[FLINK-12602][travis] Correct the flink pom artifactId config and s…

[sunjincheng121]

What is the purpose of the change

I find a shell issue in verify_scala_suffixes.sh(line 145) as follows:

grep "${module}_\d\+\.\d\+</artifactId>" "{}"

This code want to find out all modules that the module's artifactId with a scala_binary_version suffix.
but the problem is our all artifactId value is in the pattern of XXX_${scala.binary.version}, such as:

<artifactId>flink-tests_${scala.binary.version}</artifactId>

then the result always empty, so this check did not take effect.

So, we need to correct artifactId of some of the modules and correct the check logic for scala-free.

Brief change log

• remove the scala version suffix for connector-hive and queryable-state-client-java
• add the scala dependencies for table-api-scala and flink-sql-connectors
• correct the scala-free check logic in verify_scala_suffixes.sh

NOTE:
We have two ways handling of the connector:

1. Improve the script to check any (compile) dependencies with a scala-suffix(which we mentioned above).
2. Add the flink-streaming-java dependency wich provided scope for the corresponding connectors.

For approach 1, we should add check logic:

1. The command of dependency:tree should add an option: -Dincludes=org.apache.flink:_2.1:: such as org.apache.flink:flink-streaming-java_2.11.
2. The command of grep also need to add the logic: E "org.scala-lang| org.apache.flink:[^:]+_2\.1[0-9]" , also for test grep --invert-match "org.apache.flink:[^:]_2\.1[0-9]:.:.*:test".

For approach 2, we should add the dependency of link-streaming-java for flink-sql-connector-elasticsearch6 flink-sql-connector-kafka flink-sql-connector-kafka-0.10 flink-sql-connector-kafka-0.11 flink-sql-connector-kafka-0.9.

For now, I think the change logic in approach 1 is a bit complex(a lot of filtering processing logic). So, I prefer the approach 2, due to even we add some new module in the future we should well know whether we should add the scala-suffix for the artifactId, then manually add dependencies on for the scala in the pom.

So, in this PR I add the dependency of link-streaming-java for flink-sql-connectors.

Verifying this change

This change is already covered by existing tests, such as (please describe tests).

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): (no)
• The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
• The serializers: (no)
• The runtime per-record code paths (performance sensitive): (no)
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
• The S3 file system connector: (no)

Documentation

• Does this pull request introduce a new feature? (yes)
• If yes, how is the feature documented? (docs)
  1. correct the artifactId in queryable_state.md/queryable_state.zh.md.

[flinkbot]

Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.

Automated Checks

Last check on commit 3f62ecf (Tue Aug 06 15:40:56 UTC 2019)

Warnings:

• 24 pom.xml files were touched: Check for build and licensing issues.

_{Mention the bot in a comment to re-run the automated checks.}

Review Progress

• ❓ 1. The [description] looks good.
• ❓ 2. There is [consensus] that the contribution should go into to Flink.
• ❗ 3. Needs [attention] from.
  • Needs attention by @zentol [PMC]
• ❓ 4. The change fits into the overall [architecture].
• ❓ 5. Overall code [quality] is good.

Please see the Pull Request Review Guide for a full explanation of the review process.

The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot approve description to approve one or more aspects (aspects: description, consensus, architecture and quality)
• @flinkbot approve all to approve all aspects
• @flinkbot approve-until architecture to approve everything until architecture
• @flinkbot attention @username1 [@username2 ..] to require somebody's attention
• @flinkbot disapprove architecture to remove an approval you gave earlier

[sunjincheng121]

@flinkbot attention @zentol

[zentol]

this looks hacky; why can't we search the pom for dependencies with a scala-suffix instead? Shouldn't this be as simple as adding *:*_2.1*:* as an artifact pattern to -Dincludes?

[WeiZhong94]

I think this PR is a more conservative approach. Do you mean that if a flink module depends on another flink module with a scala suffix, it is also considered dependent on scala? I considered this rule, but I am not sure if it was correct. Such as a module which depends on a scala-suffix module with "provided" scope, and only uses the interfaces provided by the scala-suffix module itself, regardless of the actual scala version of the runtime. May such a module exist?

[zentol]

Do you mean that if a flink module depends on another flink module with a scala suffix, it is also considered dependent on scala?

Yes. It doesn't matter whether it is provided or not.

The only exception are dependencies with a test scope.

[sunjincheng121]

Thanks @WeiZhong94 @zentol, for @zentol's suggestion also make sense to me, just like we discuss offline @WeiZhong94 . and I mentioned above, we can take approach 1 for this PR.
I'll update the PR soon.

[sunjincheng121]

Thanks for the review @zentol @WeiZhong94 I have rebase the code and update the PR!
I appreciate if you can have a look at it again!
Best,Jincheng

[zentol]

huh.

I just found this dependency in flink-tests:

<dependency>
	<groupId>com.typesafe.akka</groupId>
	<artifactId>akka-testkit_${scala.binary.version}</artifactId>
</dependency>

This means this module does have a scala dependency, and should have a suffix. I'm wondering why the script didn't pick that up.

[flinkbot]

CI report for commit 0e36580: FAILURE Build

[sunjincheng121]

rebase code(for python CI failed)

[flinkbot]

CI report for commit 3f62ecf: SUCCESS Build

[sunjincheng121]

@zentol I have added the check of Scala suffix for all groupId, I appreciate if you have time to have look at again :)!

[sunjincheng121]

Merging...