[FLINK-21300][docs] Update module-related documentation #15230

LadyForest · 2021-03-16T08:06:20Z

What is the purpose of the change

This pull request is a subtask of FLINK-21045 to update the docs on module-related changes.

Brief change log

Update the overview.md, add links to LOAD and UNLOAD statement, update the reserved keywords(add MODULES, the reason is [FLINK-21298][table] Support 'USE MODULES' syntax both in SQL parser, TableEnvironment and SQL CLI #15005 (comment)).
Add load.md and unload.md to describe LOAD MODULE and UNLOAD MODULE usage.
Update the use.md (add description of USE MODULES) and show.md(add description of SHOW MODULES/SHOW FULL MODULES)
Update the modules.md, add the description on module lifecycle, resolution order change etc.
Update Python doctable_envirronment.md, add links to use_modules and list_full_modules
Update related Chinese doc(without translation because I think it's better to make it a separate task to ease review)

Verifying this change

This change can be verified after rebuilding docs. I've updated myself to adapt FLINK-21774 ~~Please note that at this moment, the output of SHOW [FULL] MODULES stills keeps table schema even if the output is empty, this should be updated together with FLINK-21774~~.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

flinkbot · 2021-03-16T08:09:30Z

Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.

Automated Checks

Last check on commit 569104d (Tue Mar 16 08:09:29 UTC 2021)

✅no warnings

_{Mention the bot in a comment to re-run the automated checks.}

Review Progress

❓ 1. The [description] looks good.
❓ 2. There is [consensus] that the contribution should go into to Flink.
❓ 3. Needs [attention] from.
❓ 4. The change fits into the overall [architecture].
❓ 5. Overall code [quality] is good.

Please see the Pull Request Review Guide for a full explanation of the review process.

The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot approve description to approve one or more aspects (aspects: description, consensus, architecture and quality)
@flinkbot approve all to approve all aspects
@flinkbot approve-until architecture to approve everything until architecture
@flinkbot attention @username1 [@username2 ..] to require somebody's attention
@flinkbot disapprove architecture to remove an approval you gave earlier

LadyForest · 2021-03-16T08:24:45Z

I'm also fine with updating myself if #15213 gets merged firstly.

flinkbot · 2021-03-16T08:28:15Z

CI report:

2f99abe Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run travis re-run the last Travis build
@flinkbot run azure re-run the last Azure build

LadyForest · 2021-03-16T12:05:22Z

Hi @wuchong, could you mind taking a look at this PR?

LadyForest · 2021-03-17T15:50:35Z

I'm also fine with updating myself if #15213 gets merged firstly.

done

wuchong · 2021-03-20T15:34:54Z

docs/content/docs/dev/table/sql/load.md

+-- +-------------+
+-- | module name |
+-- +-------------+
+-- | core |
+-- | hive |
+-- +-------------+


There is no -- prefix printed in SQL Client.

wuchong · 2021-03-20T15:35:20Z

docs/content/docs/dev/table/sql/load.md

+
+The following grammar gives an overview of the available syntax:
+```sql
+LOAD MODULE `module_name` [WITH ('key1' = 'val1', 'key2' = 'val2', ...)]


nit: should we explain what's the key1 val1 properties?

Besides, users don't need always escape the module_name.

should we explain what's the key1 val1 properties?

Yes, I think we'd better add a description to let users know how to design their own module and module factory impl.

wuchong · 2021-03-20T15:44:30Z

docs/content.zh/docs/dev/table/sql/unload.md

+[INFO] Unload module succeeded!
+
+Flink SQL> SHOW MODULES;
+-- Empty set


wuchong · 2021-03-20T15:45:41Z

docs/content.zh/docs/dev/table/sql/unload.md

+
+The following grammar gives an overview of the available syntax:
+```sql
+UNLOAD MODULE `module_name`


Users don't need always escape the module_name. UNLOAD MODULE module_name is fine.

wuchong · 2021-03-20T15:47:30Z

docs/content.zh/docs/dev/table/sql/use.md

@@ -215,6 +251,11 @@ USE CATALOG catalog_name

 设置当前的 catalog。所有后续命令未显式指定 catalog 的将使用此 catalog。如果指定的的 catalog 不存在，则抛出异常。默认的当前 catalog 是 `default_catalog`。

+## USE MODULES
+```sql
+USE MODULES `module_name1`[, `module_name2`, ...]


wuchong · 2021-03-20T15:50:58Z

docs/content.zh/docs/dev/table/sql/show.md

+
+展示当前环境激活的所有 module。
+
+## SHOW FULL MODULES


I suggest to merge SHOW FULL MODUELS into SHOW MODULES, just like what we do for SHOW [USER] FUNCTIONS.

Thanks @LadyForest for the detailed documentation. It already looks in a good shape. I only left some minor comments.

Many thanks to @wuchong for the thorough review. I'll address your comments right now.

wuchong · 2021-03-20T15:51:31Z

docs/content.zh/docs/dev/table/sql/show.md

@@ -38,6 +38,8 @@ SHOW 语句用于列出所有的 catalog，或者列出当前 catalog 中所有
 - SHOW TABLES
 - SHOW VIEWS
 - SHOW FUNCTIONS
+- SHOW MODULES
+- SHOW FULL MODULES


wuchong · 2021-03-20T15:51:56Z

docs/content.zh/docs/dev/table/sql/load.md

+-- +-------------+
+-- | module name |
+-- +-------------+
+-- | core |
+-- | hive |
+-- +-------------+


wuchong · 2021-03-20T15:52:05Z

docs/content.zh/docs/dev/table/sql/load.md

+
+The following grammar gives an overview of the available syntax:
+```sql
+LOAD MODULE `module_name` [WITH ('key1' = 'val1', 'key2' = 'val2', ...)]


wuchong · 2021-03-20T15:56:40Z

docs/content.zh/docs/dev/table/modules.md


-Objects provided by modules are considered part of Flink's system (built-in) objects; thus, they don't have any namespaces.
+A module can be loaded, enabled, disabled and unloaded. When TableEnvironment loads a module initially, it enables the module by default. Flink supports multiple modules and keeps track of the loading order to resolve metadata. *E.g.*, when there are two functions of the same name residing in two modules, Flink always resolves the function reference to the one in the 1st enabled module.


to the one in the 1st enabled module.

This sounds we always resolve function by only looking up the first module which is incorrect.

to the one in the 1st enabled module.

This sounds we always resolve function by only looking up the first module which is incorrect.

Yes, it's somewhat misleading. While this is the original statement before I made this change. I'll do the rewrite to make it more accurate.

wuchong · 2021-03-20T16:06:48Z

Thanks @LadyForest for the detailed documentation. It already looks in a good shape. I only left some minor comments.

LadyForest · 2021-03-22T16:10:32Z

Hi @wuchong, I updated this PR according to your suggestions. Would you mind taking another look when you're free?

wuchong

LGTM.

rmetzger added review=description? component=Documentation component=TableSQL/API component=TableSQL/Client labels Mar 16, 2021

LadyForest added 2 commits March 17, 2021 23:27

[FLINK-21300][docs] Update module-related documentation

49ad82f

Rebase master and adapt changes made by FLINK-21774

75810a4

LadyForest force-pushed the FLINK-21300 branch from 569104d to 75810a4 Compare March 17, 2021 15:36

wuchong reviewed Mar 20, 2021

View reviewed changes

Address comments.

2f99abe

wuchong approved these changes Mar 23, 2021

View reviewed changes

wuchong merged commit 15b5697 into apache:master Mar 23, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FLINK-21300][docs] Update module-related documentation #15230

[FLINK-21300][docs] Update module-related documentation #15230

LadyForest commented Mar 16, 2021 •

edited

Loading

flinkbot commented Mar 16, 2021

LadyForest commented Mar 16, 2021

flinkbot commented Mar 16, 2021 •

edited

Loading

LadyForest commented Mar 16, 2021

LadyForest commented Mar 17, 2021

wuchong Mar 20, 2021

wuchong Mar 20, 2021

wuchong Mar 20, 2021

LadyForest Mar 21, 2021

wuchong Mar 20, 2021

wuchong Mar 20, 2021

wuchong Mar 20, 2021

wuchong Mar 20, 2021

LadyForest Mar 21, 2021

wuchong Mar 20, 2021

wuchong Mar 20, 2021

wuchong Mar 20, 2021

wuchong Mar 20, 2021 •

edited

Loading

LadyForest Mar 21, 2021

wuchong commented Mar 20, 2021

LadyForest commented Mar 22, 2021

wuchong left a comment


		展示当前环境激活的所有 module。

		## SHOW FULL MODULES


		Objects provided by modules are considered part of Flink's system (built-in) objects; thus, they don't have any namespaces.
		A module can be loaded, enabled, disabled and unloaded. When TableEnvironment loads a module initially, it enables the module by default. Flink supports multiple modules and keeps track of the loading order to resolve metadata. E.g., when there are two functions of the same name residing in two modules, Flink always resolves the function reference to the one in the 1st enabled module.

[FLINK-21300][docs] Update module-related documentation #15230

[FLINK-21300][docs] Update module-related documentation #15230

Conversation

LadyForest commented Mar 16, 2021 • edited Loading

What is the purpose of the change

Brief change log

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

flinkbot commented Mar 16, 2021

Automated Checks

Review Progress

LadyForest commented Mar 16, 2021

flinkbot commented Mar 16, 2021 • edited Loading

CI report:

LadyForest commented Mar 16, 2021

LadyForest commented Mar 17, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wuchong Mar 20, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wuchong commented Mar 20, 2021

LadyForest commented Mar 22, 2021

wuchong left a comment

Choose a reason for hiding this comment

LadyForest commented Mar 16, 2021 •

edited

Loading

flinkbot commented Mar 16, 2021 •

edited

Loading

wuchong Mar 20, 2021 •

edited

Loading