Skip to content

Commit

Permalink
[FLINK-14907] [docs] update documentation on credential configuration…
Browse files Browse the repository at this point in the history
… for ABS
  • Loading branch information
knaufk committed Dec 6, 2019
1 parent e24416e commit 3743286
Showing 1 changed file with 13 additions and 6 deletions.
19 changes: 13 additions & 6 deletions docs/ops/filesystems/azure.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,17 +62,24 @@ cp ./opt/flink-azure-fs-hadoop-{{ site.version }}.jar ./plugins/azure-fs-hadoop/

`flink-azure-fs-hadoop` registers default FileSystem wrappers for URIs with the *wasb:https://* and *wasbs:https://* (SSL encrypted access) scheme.

#### Configurations setup
After setting up the Azure Blob Storage FileSystem wrapper, you need to configure credentials to make sure that Flink is allowed to access Azure Blob Storage.
### Credentials Configuration

To allow for easy adoption, you can use the same configuration keys in `flink-conf.yaml` as in Hadoop's `core-site.xml`
Hadoop's Azure Filesystem supports configuration of credentials via the Hadoop configuration as
outlined in the [Hadoop Azure Blob Storage documentation](https://hadoop.apache.org/docs/current/hadoop-azure/index.html#Configuring_Credentials).
For convenience Flink forwards all Flink configurations with a key prefix of `fs.azure` to the
Hadoop configuration of the filesystem. Consequentially, the azure blob storage key can be configured
in `flink-conf.yaml` via:

You can see the configuration keys in the [Hadoop Azure Blob Storage documentation](https://hadoop.apache.org/docs/current/hadoop-azure/index.html#Configuring_Credentials).
{% highlight yaml %}
fs.azure.account.key.<account_name>.blob.core.windows.net: <azure_storage_key>
{% endhighlight %}

There are some required configurations that must be added to `flink-conf.yaml`:
Alternatively, the the filesystem can be configured to read the Azure Blob Storage key from an
environment variable `AZURE_STORAGE_KEY` by setting the following configuration keys in
`flink-conf.yaml`.

{% highlight yaml %}
fs.azure.account.key.youraccount.blob.core.windows.net: Azure Blob Storage access key
fs.azure.account.keyprovider.<account_name>.blob.core.windows.net: org.apache.flink.fs.azurefs.EnvironmentVariableKeyProvider
{% endhighlight %}

{% top %}

0 comments on commit 3743286

Please sign in to comment.