Skip to content

Commit

Permalink
[FLINK-20426][docs] Fix broken links after Hadoop page removal
Browse files Browse the repository at this point in the history
This closes apache#14272
  • Loading branch information
rmetzger committed Dec 1, 2020
1 parent 0a354c7 commit 8b2e4c2
Show file tree
Hide file tree
Showing 6 changed files with 32 additions and 9 deletions.
12 changes: 11 additions & 1 deletion docs/dev/batch/hadoop_compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,17 @@ and Reducers.
</dependency>
{% endhighlight %}

See also **[how to configure hadoop dependencies]({{ site.baseurl }}/deployment/resource-providers/hadoop.html#add-hadoop-classpaths)**.
If you want to run your Flink application locally (from your IDE), you also need to add
a `hadoop-client` dependency such as:

{% highlight xml %}
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>2.8.3</version>
<scope>provided</scope>
</dependency>
{% endhighlight %}

### Using Hadoop InputFormats

Expand Down
9 changes: 8 additions & 1 deletion docs/dev/batch/hadoop_compatibility.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,14 @@ and Reducers.
</dependency>
{% endhighlight %}

See also **[how to configure hadoop dependencies]({{ site.baseurl }}/deployment/resource-providers/hadoop.html#add-hadoop-classpaths)**.
{% highlight xml %}
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>2.8.3</version>
<scope>provided</scope>
</dependency>
{% endhighlight %}

### Using Hadoop InputFormats

Expand Down
4 changes: 2 additions & 2 deletions docs/dev/project-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,8 +152,8 @@ for details on how to build Flink for a specific Scala version.
*(The only exception being when using existing Hadoop input-/output formats with Flink's Hadoop compatibility wrappers)*

If you want to use Flink with Hadoop, you need to have a Flink setup that includes the Hadoop dependencies, rather than
adding Hadoop as an application dependency. Please refer to the [Hadoop Setup Guide]({{ site.baseurl }}/deployment/resource-providers/hadoop.html)
for details.
adding Hadoop as an application dependency. Flink will use the Hadoop dependencies specified by the `HADOOP_CLASSPATH`
environment variable, which can usually be set by calling `export HADOOP_CLASSPATH=``hadoop classpath```

There are two main reasons for that design:

Expand Down
4 changes: 2 additions & 2 deletions docs/dev/project-configuration.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,8 +152,8 @@ for details on how to build Flink for a specific Scala version.
*(The only exception being when using existing Hadoop input-/output formats with Flink's Hadoop compatibility wrappers)*

If you want to use Flink with Hadoop, you need to have a Flink setup that includes the Hadoop dependencies, rather than
adding Hadoop as an application dependency. Please refer to the [Hadoop Setup Guide]({{ site.baseurl }}/deployment/resource-providers/hadoop.html)
for details.
adding Hadoop as an application dependency. Flink will use the Hadoop dependencies specified by the `HADOOP_CLASSPATH`
environment variable, which can usually be set by calling `export HADOOP_CLASSPATH=``hadoop classpath```

There are two main reasons for that design:

Expand Down
7 changes: 5 additions & 2 deletions docs/dev/table/connectors/hive/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,8 +92,11 @@ to make the integration work in Table API program or SQL in SQL Client.
Alternatively, you can put these dependencies in a dedicated folder, and add them to classpath with the `-C`
or `-l` option for Table API program or SQL Client respectively.

Apache Hive is built on Hadoop, so you need Hadoop dependency first, please refer to
[Providing Hadoop classes]({{ site.baseurl }}/deployment/resource-providers/hadoop.html#providing-hadoop-classes).
Apache Hive is built on Hadoop, so you need to provide Hadoop dependenies, by setting the `HADOOP_CLASSPATH`
environment variable:
```
export HADOOP_CLASSPATH=`hadoop classpath`
```

There are two ways to add Hive dependencies. First is to use Flink's bundled Hive jars. You can choose a bundled Hive jar according to the version of the metastore you use. Second is to add each of the required jars separately. The second way can be useful if the Hive version you're using is not listed here.

Expand Down
5 changes: 4 additions & 1 deletion docs/dev/table/connectors/hive/index.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,10 @@ Flink 支持一下的 Hive 版本。
或者,您可以将这些依赖项放在专用文件夹中,并分别使用 Table API 程序或 SQL Client 的`-C``-l`选项将它们添加到 classpath 中。

Apache Hive 是基于 Hadoop 之上构建的, 首先您需要 Hadoop 的依赖,请参考
[Providing Hadoop classes]({{ site.baseurl }}/zh/deployment/resource-providers/hadoop.html#providing-hadoop-classes).
Providing Hadoop classes:
```
export HADOOP_CLASSPATH=`hadoop classpath`
```

有两种添加 Hive 依赖项的方法。第一种是使用 Flink 提供的 Hive Jar包。您可以根据使用的 Metastore 的版本来选择对应的 Hive jar。第二个方式是分别添加每个所需的 jar 包。如果您使用的 Hive 版本尚未在此处列出,则第二种方法会更适合。

Expand Down

0 comments on commit 8b2e4c2

Please sign in to comment.