Skip to content

Commit

Permalink
[hotfix][docs] Improve data type documentation for Scala users
Browse files Browse the repository at this point in the history
  • Loading branch information
twalthr committed Jul 9, 2020
1 parent ed5fc5c commit 7d52ddc
Show file tree
Hide file tree
Showing 3 changed files with 23 additions and 5 deletions.
8 changes: 7 additions & 1 deletion docs/dev/table/functions/udfs.md
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,10 @@ Regular JVM method calling semantics apply. Therefore, it is possible to:
- use object inheritance such as `eval(Object)` that takes both `LocalDateTime` and `Integer`,
- and combinations of the above such as `eval(Object...)` that takes all kinds of arguments.

If you intend to implement functions in Scala, please add the `scala.annotation.varargs` annotation in
case of variable arguments. Furthermore, it is recommended to use boxed primitives (e.g. `java.lang.Integer`
instead of `Int`) to support `NULL`.

The following snippets shows an example of an overloaded function:

<div class="codetabs" markdown="1">
Expand Down Expand Up @@ -240,6 +244,8 @@ public static class SumFunction extends ScalarFunction {
<div data-lang="Scala" markdown="1">
{% highlight scala %}
import org.apache.flink.table.functions.ScalarFunction
import java.lang.Integer
import java.lang.Double
import scala.annotation.varargs

// function with overloaded evaluation methods
Expand Down Expand Up @@ -280,7 +286,7 @@ If more advanced type inference logic is required, an implementer can explicitly

The automatic type inference inspects the function's class and evaluation methods to derive data types for the arguments and result of a function. `@DataTypeHint` and `@FunctionHint` annotations support the automatic extraction.

For a full list of classes that can be implicitly mapped to a data type, see the [data type section]({% link dev/table/types.md %}#data-type-annotations).
For a full list of classes that can be implicitly mapped to a data type, see the [data type extraction section]({% link dev/table/types.md %}#data-type-extraction).

**`@DataTypeHint`**

Expand Down
12 changes: 9 additions & 3 deletions docs/dev/table/types.md
Original file line number Diff line number Diff line change
Expand Up @@ -1348,15 +1348,21 @@ DataTypes.NULL()
|`java.lang.Object` | X | X | *Default* |
|*any class* | | (X) | Any non-primitive type. |

Data Type Annotations
---------------------
Data Type Extraction
--------------------

At many locations in the API, Flink tries to automatically extract data type from class information using
reflection to avoid repetitive manual schema work. However, extracting a data type reflectively is not always
successful because logical information might be missing. Therefore, it might be necessary to add additional
information close to a class or field declaration for supporting the extraction logic.

The following table lists classes that can be implicitly mapped to a data type without requiring further information:
The following table lists classes that can be implicitly mapped to a data type without requiring further information.

If you intend to implement classes in Scala, *it is recommended to use boxed types* (e.g. `java.lang.Integer`)
instead of Scala's primitives. Scala's primitives (e.g. `Int` or `Double`) are compiled to JVM primitives (e.g.
`int`/`double`) and result in `NOT NULL` semantics as shown in the table below. Furthermore, Scala primitives that
are used in generics (e.g. `java.lang.Map[Int, Double]`) are erased during compilation and lead to class
information similar to `java.lang.Map[java.lang.Object, java.lang.Object]`.

| Class | Data Type |
|:----------------------------|:------------------------------------|
Expand Down
8 changes: 7 additions & 1 deletion docs/dev/table/types.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -1245,7 +1245,13 @@ DataTypes.NULL()

Flink API 经常尝试使用反射自动从类信息中提取数据类型,以避免重复的手动定义模式工作。然而以反射方式提取数据类型并不总是成功的,因为可能会丢失逻辑信息。因此,可能有必要在类或字段声明附近添加额外信息以支持提取逻辑。

下表列出了可以隐式映射到数据类型而无需额外信息的类:
下表列出了可以隐式映射到数据类型而无需额外信息的类。

If you intend to implement classes in Scala, *it is recommended to use boxed types* (e.g. `java.lang.Integer`)
instead of Scala's primitives. Scala's primitives (e.g. `Int` or `Double`) are compiled to JVM primitives (e.g.
`int`/`double`) and result in `NOT NULL` semantics as shown in the table below. Furthermore, Scala primitives that
are used in generics (e.g. `java.lang.Map[Int, Double]`) are erased during compilation and lead to class
information similar to `java.lang.Map[java.lang.Object, java.lang.Object]`.

|| 数据类型 |
|:----------------------------|:------------------------------------|
Expand Down

0 comments on commit 7d52ddc

Please sign in to comment.