Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MinIO Server releases and other fixes #1172

Merged
merged 5 commits into from
Mar 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Tag-Based Policy Conditions

.. versionchanged:: RELEASE.2022-10-02T19-29-29Z

Policies can use conditions to limit a user's access only to objects with a specific tag.
Policies can use conditions to limit a user's access only to objects with a :ref:`specific tag <minio-object-tagging>`.

MinIO supports :s3-docs:`tag-based conditionals <tagging-and-policies.html>` for policies for :ref:`selected actions <minio-selected-conditional-actions>`.
Use the ``s3:ExistingObjectTag/<key>`` in the ``Condition`` statement of the policy.
Expand Down Expand Up @@ -156,11 +156,11 @@ Policy Document Structure
MinIO policy documents use the same schema as
:aws-docs:`AWS IAM Policy <IAM/latest/UserGuide/access.html>` documents.

The following sample document provides a template for creating custom
policies for use with a MinIO deployment. For more complete documentation on IAM
policy elements, see the :aws-docs:`IAM JSON Policy Elements Reference
<IAM/latest/UserGuide/reference_policies_elements.html>`.
The maximum size for a policy document is 2048 characters.
The following sample document provides a template for creating custom policies for use with a MinIO deployment.
For more complete documentation on IAM policy elements, see the :aws-docs:`IAM JSON Policy Elements Reference <IAM/latest/UserGuide/reference_policies_elements.html>`.

The maximum size for any single policy document is 20KiB.
There is no limit to the number of policy documents that can be attached to a user or group.

.. code-block:: javascript
:class: copyable
Expand Down
13 changes: 13 additions & 0 deletions source/administration/object-management.rst
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,19 @@ The specific client behavior on write, list, get, or delete operations on a buck

See :ref:`minio-bucket-versioning` for more complete documentation.

.. _minio-object-tagging:

Object Tagging
--------------

MinIO supports adding custom tags to an object.
A tag is a key-value pair included in the metadata of an object.
Tags can be used to control access with policies or locate an object with :mc-cmd:`mc find --tags`.

MinIO supports adding up to 10 custom tags to an object.

For more on setting tags, refer to :mc:`mc tag set`.

Object Retention
----------------

Expand Down
117 changes: 25 additions & 92 deletions source/administration/server-side-encryption.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,46 +15,36 @@ Server-Side Encryption of Objects
.. |SSE| replace:: :abbr:`SSE (Server-Side Encryption)`
.. |KMS| replace:: :abbr:`KMS (Key Management System)`

MinIO Server-Side Encryption (SSE) protects objects as part of write operations,
allowing clients to take advantage of server processing power to secure objects
at the storage layer (encryption-at-rest). SSE also provides key functionality
to regulatory and compliance requirements around secure locking and erasure.
MinIO Server-Side Encryption (SSE) protects objects as part of write operations, allowing clients to take advantage of server processing power to secure objects at the storage layer (encryption-at-rest).
SSE also provides key functionality to regulatory and compliance requirements around secure locking and erasure.

MinIO SSE uses the :minio-git:`MinIO Key Encryption Service (KES) <kes>` and an
external Key Management Service (KMS) for performing secured cryptographic
operations at scale. MinIO also supports client-managed key management, where
the application takes full responsibility for creating and managing encryption
keys for use with MinIO SSE.
MinIO SSE uses the :minio-git:`MinIO Key Encryption Service (KES) <kes>` and an external Key Management Service (KMS) for performing secured cryptographic operations at scale.
MinIO also supports client-managed key management, where the application takes full responsibility for creating and managing encryption keys for use with MinIO SSE.

MinIO SSE is feature and API compatible with
:s3-docs:`AWS Server-Side Encryption <server-side-encryption.html>` and
supports the following encryption strategies:
MinIO SSE is feature and API compatible with :s3-docs:`AWS Server-Side Encryption <server-side-encryption.html>` and supports the following encryption strategies:

.. tab-set::

.. tab-item:: SSE-KMS *Recommended*
:sync: sse-kms

MinIO supports enabling automatic SSE-KMS encryption of all objects
written to a bucket using a specific External Key (EK) stored on the
external |KMS|. Clients can override the bucket-default |EK| by specifying
an explicit key as part of the write operation.
MinIO supports enabling automatic SSE-KMS encryption of all objects written to a bucket using a specific External Key (EK) stored on the external |KMS|.
Clients can override the bucket-default |EK| by specifying an explicit key as part of the write operation.

For buckets without automatic SSE-KMS encryption, clients can specify
an |EK| as part of the write operation instead.
For buckets without automatic SSE-KMS encryption, clients can specify an |EK| as part of the write operation instead.

SSE-KMS provides more granular and customizable encryption compared to
SSE-S3 and SSE-C and is recommended over the other supported encryption
methods.
SSE-KMS provides more granular and customizable encryption compared to SSE-S3 and SSE-C and is recommended over the other supported encryption methods.

For a tutorial on enabling SSE-KMS in a local (non-production) MinIO
Deployment, see :ref:`minio-encryption-sse-kms-quickstart`. For
production MinIO deployments, use one of the following guides:
For a tutorial on enabling SSE-KMS in a local (non-production) MinIO Deployment, see :ref:`minio-encryption-sse-kms-quickstart`.
For production MinIO deployments, use one of the following guides:

- :ref:`AWS SecretsManager <minio-sse-aws>`
- :ref:`Google Cloud SecretManager <minio-sse-gcp>`
- :ref:`Azure Key Vault <minio-sse-azure>`
- :ref:`Hashicorp KeyVault <minio-sse-vault>`
- :kes-docs:`AWS Secrets Manager <integrations/aws-secrets-manager/>`
- :kes-docs:`Azure Key Vault <integrations/azure-keyvault/>`
- :kes-docs:`Entrust KeyControl <integrations/entrust-keycontrol/>`
- :kes-docs:`Fortanix SDKMS <integrations/fortanix-sdkms/>`
- :kes-docs:`Google Cloud Secret Manager <integrations/google-cloud-secret-manager/>`
- :kes-docs:`Hashicorp Vault Keystore <integrations/hashicorp-vault-keystore/>`
- :kes-docs:`Thales CipherTrust Manager (formerly Gemalto KeySecure) <integrations/thales-ciphertrust/>`

.. tab-item:: SSE-S3
:sync: sse-s3
Expand All @@ -70,10 +60,13 @@ supports the following encryption strategies:
Deployment, see :ref:`minio-encryption-sse-s3-quickstart`. For
production MinIO deployments, use one of the following guides:

- :ref:`AWS SecretsManager <minio-sse-aws>`
- :ref:`Google Cloud SecretManager <minio-sse-gcp>`
- :ref:`Azure Key Vault <minio-sse-azure>`
- :ref:`Hashicorp KeyVault <minio-sse-vault>`
- :kes-docs:`AWS Secrets Manager <integrations/aws-secrets-manager/>`
- :kes-docs:`Azure Key Vault <integrations/azure-keyvault/>`
- :kes-docs:`Entrust KeyControl <integrations/entrust-keycontrol/>`
- :kes-docs:`Fortanix SDKMS <integrations/fortanix-sdkms/>`
- :kes-docs:`Google Cloud Secret Manager <integrations/google-cloud-secret-manager/>`
- :kes-docs:`Hashicorp Vault Keystore <integrations/hashicorp-vault-keystore/>`
- :kes-docs:`Thales CipherTrust Manager (formerly Gemalto KeySecure) <integrations/thales-ciphertrust/>`

.. tab-item:: SSE-C
:sync: sse-c
Expand Down Expand Up @@ -126,66 +119,6 @@ For more information, see:
- :ref:`SSE-C Secure Erasure and Locking
<minio-encryption-sse-c-erasure-locking>`

Encryption Internals
--------------------

.. note::

The following section describes MinIO internal logic and functionality.
This information is purely educational and is not necessary for
configuring or implementing any MinIO feature.

.. _minio-encryption-sse-content-encryption:

Content Encryption
~~~~~~~~~~~~~~~~~~

The MinIO server uses an authenticated encryption scheme
(:ref:`AEAD <minio-encryption-sse-primitives>`) to en/decrypt and authenticate
the object content. The AEAD is combined with some state to build a
**Secure Channel**. A Secure Channel is a cryptographic construction that
ensures confidentiality and integrity of the processed data. In particular, the
Secure Channel splits the plaintext content into fixed size chunks and
en/decrypts each chunk separately using an unique key-nonce combination.

The following text diagram illustrates Secure Channel Construction of an
encrypted object:

The Secure Channel splits the object content into chunks of a fixed size of
``65536`` bytes. The last chunk may be smaller to avoid adding additional
overhead and is treated specially to prevent truncation attacks. The nonce
value is ``96`` bits long and generated randomly per object / multi-part part.
The Secure Channel supports plaintexts up to ``65536 * 2^32 = 256 TiB``.

For S3 multi-part operations, each object part is en/decrypted with the Secure
Channel Construction scheme shown above. For each part, MinIO generates a secret
key derived from the Object Encryption Key (OEK) and the part number using a
pseudo-random function (:ref:`PRF <minio-encryption-sse-primitives>`), such that
``key = PRF(OEK, part_id)``.

.. _minio-encryption-sse-primitives:

Cryptographic Primitives
~~~~~~~~~~~~~~~~~~~~~~~~

The MinIO server uses the following cryptographic primitive implementations:

.. list-table::
:header-rows: 1
:widths: 40 60
:width: 100%

* -
- Primitives

* - Pseudo-Random Functions (PRF)
- HMAC-SHA-256

* - :ref:`AEAD <minio-encryption-sse-content-encryption>`
- ``ChaCha20-Poly1305`` by default.

``AES-256-GCM`` for x86-64 CPUs with the AES-NI extension.

.. toctree::
:titlesonly:
:hidden:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,16 +49,15 @@ You can enable bucket-default SSE-KMS encryption using the
- Replace ``play/mybucket`` with the :mc:`alias <mc alias>` and bucket
on which you want to enable automatic SSE-KMS encryption.

MinIO SSE-KMS is functionally compatible with AWS S3
:s3-docs:`Server-Side Encryption with KMS keys stored in AWS
<UsingKMSEncryption.html>` while expanding support to include the
following KMS providers:
MinIO SSE-KMS is functionally compatible with AWS S3 :s3-docs:`Server-Side Encryption with KMS keys stored in AWS <UsingKMSEncryption.html>` while expanding support to include the following KMS providers:

- :ref:`AWS SecretsManager <minio-sse-aws>`
- :ref:`Google Cloud SecretManager <minio-sse-gcp>`
- :ref:`Azure Key Vault <minio-sse-azure>`
- :ref:`Hashicorp KeyVault <minio-sse-vault>`
- Thales CipherTrust (formerly Gemalto KeySecure)
- :kes-docs:`AWS Secrets Manager <integrations/aws-secrets-manager/>`
- :kes-docs:`Azure Key Vault <integrations/azure-keyvault/>`
- :kes-docs:`Entrust KeyControl <integrations/entrust-keycontrol/>`
- :kes-docs:`Fortanix SDKMS <integrations/fortanix-sdkms/>`
- :kes-docs:`Google Cloud Secret Manager <integrations/google-cloud-secret-manager/>`
- :kes-docs:`Hashicorp Vault Keystore <integrations/hashicorp-vault-keystore/>`
- :kes-docs:`Thales CipherTrust Manager (formerly Gemalto KeySecure) <integrations/thales-ciphertrust/>`

.. _minio-encryption-sse-kms-quickstart:

Expand All @@ -71,10 +70,13 @@ supporting |SSE| with SSE-KMS in evaluation and early development environments.
For extended development or production environments, use one of the following
supported external Key Management Services (KMS):

- :ref:`AWS SecretsManager <minio-sse-aws>`
- :ref:`Google Cloud SecretManager <minio-sse-gcp>`
- :ref:`Azure Key Vault <minio-sse-azure>`
- :ref:`Hashicorp KeyVault <minio-sse-vault>`
- :kes-docs:`AWS Secrets Manager <integrations/aws-secrets-manager/>`
- :kes-docs:`Azure Key Vault <integrations/azure-keyvault/>`
- :kes-docs:`Entrust KeyControl <integrations/entrust-keycontrol/>`
- :kes-docs:`Fortanix SDKMS <integrations/fortanix-sdkms/>`
- :kes-docs:`Google Cloud Secret Manager <integrations/google-cloud-secret-manager/>`
- :kes-docs:`Hashicorp Vault Keystore <integrations/hashicorp-vault-keystore/>`
- :kes-docs:`Thales CipherTrust Manager (formerly Gemalto KeySecure) <integrations/thales-ciphertrust/>`

.. include:: /includes/common/common-minio-kes.rst
:start-after: start-kes-play-sandbox-warning
Expand Down
Binary file added source/images/grafana-node.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
26 changes: 10 additions & 16 deletions source/includes/common-minio-sse.rst
Original file line number Diff line number Diff line change
@@ -1,32 +1,26 @@
.. start-sse-dek

MinIO generates a Data Encryption Key (DEK) using the |EK|. Specifically,
:minio-git:`MinIO Key Encryption Service (KES) <kes>` requests a new
cryptographic key from the KMS using the |EK| as the "root" key.
MinIO generates a Data Encryption Key (DEK) using the |EK|.
Specifically, :minio-git:`MinIO Key Encryption Service (KES) <kes>` requests a new cryptographic key from the KMS using the |EK| as the "root" key.

KES returns both the plain-text *and* an |EK|-encrypted representation of the
DEK. MinIO stores the encrypted representation as part of the object metadata.
KES returns both the plain-text *and* an |EK|-encrypted representation of the DEK.
MinIO stores the encrypted representation as part of the object metadata.

.. end-sse-dek

.. start-sse-kek

MinIO uses a deterministic algorithm to generate a 256-bit unique Key
Encryption Key (KEK). The key-derivation algorithm uses a pseudo-random function
(:ref:`PRF <minio-encryption-sse-primitives>`) that takes the plain-text |DEK|,
a randomly generated initialization vector, and a context consisting of values
like the bucket and object name.
MinIO uses a deterministic algorithm to generate a 256-bit unique Key Encryption Key (KEK).
The key-derivation algorithm uses a pseudo-random function that takes the plain-text |DEK|, a randomly generated initialization vector, and a context consisting of values like the bucket and object name.

MinIO generates the KEK at the time of each cryptographic encryption or
decryption operation and *never* stores the KEK to a drive.
MinIO generates the KEK at the time of each cryptographic encryption or decryption operation and *never* stores the KEK to a drive.

.. end-sse-kek

.. start-sse-oek

MinIO generates a random 256-bit unique Object Encryption Key (OEK) and uses
that key to encrypt the object. MinIO never stores the plaintext representation
of the OEK on a drive. The plaintext OEK resides in RAM during cryptographic
operations.
MinIO generates a random 256-bit unique Object Encryption Key (OEK) and uses that key to encrypt the object.
MinIO never stores the plaintext representation of the OEK on a drive.
The plaintext OEK resides in RAM during cryptographic operations.

.. end-sse-oek
25 changes: 19 additions & 6 deletions source/operations/monitoring/grafana.rst
Original file line number Diff line number Diff line change
Expand Up @@ -56,23 +56,36 @@ MinIO Bucket Metrics Dashboard

Visualize MinIO bucket metrics with the official MinIO Grafana dashboard for buckets available on the `Grafana dashboard portal <https://grafana.com/grafana/dashboards/19237-minio-bucket-dashboard/>`__.

Bucket metrics can be viewed in the Grafana dashboard using the `bucket JSON file on GitHub <https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/minio-bucket.json>`__.
Bucket metrics can be viewed in the Grafana dashboard using the `bucket JSON file on GitHub <https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/bucket/minio-bucket.json>`__.

.. image:: /images/grafana-bucket.png
:width: 600px
:alt: A sample of the MinIO Grafana dashboard showing many different captured metrics for MinIO buckets.
:align: center

.. _minio-node-grafana-metrics:

MinIO Node Metrics Dashboard
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Node metrics can be viewed in the Grafana dashboard using the `node JSON file on GitHub <https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/node/minio-node.json>`__.

.. image:: /images/grafana-node.png
:width: 600px
:alt: A sample of the MinIO Grafana dashboard showing many different captured metrics for MinIO nodes.
:align: center


.. _minio-replication-grafana-metrics:

MinIO Cluster Replication Metrics Dashboard
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO Replication Metrics Dashboard
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Visualize MinIO bucket metrics with the official MinIO Grafana dashboard for cluster replication available on the `Grafana dashboard portal <https://grafana.com/grafana/dashboards/15305-minio-cluster-replication-dashboard/>`__.
Visualize MinIO bucket metrics with the official MinIO Grafana dashboard for replication available on the `Grafana dashboard portal <https://grafana.com/grafana/dashboards/15305-minio-replication-dashboard/>`__.

Cluster replication metrics can be viewed in the Grafana dashboard using the `cluster replication JSON file on GitHub <https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/minio-replication.json>`__.
Cluster replication metrics can be viewed in the Grafana dashboard using the `cluster replication JSON file on GitHub <https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/replication/minio-replication.json>`__.

.. image:: /images/grafana-replication.png
:width: 600px
:alt: A sample of the MinIO Grafana dashboard showing many different captured metrics for cluster replication.
:alt: A sample of the MinIO Grafana dashboard showing many different captured metrics for replication.
:align: center
1 change: 1 addition & 0 deletions source/reference/minio-mc/mc-pipe.rst
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,7 @@ Set Tags on Uploaded Objects
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The following command creates an object on a MinIO deployment with an ALIAS of ``myminio`` in bucket ``mybucket`` with two tags.
You can set up to 10 tags on an object.

.. code-block:: shell
:class: copyable
Expand Down
2 changes: 2 additions & 0 deletions source/reference/minio-mc/mc-tag-set.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ The :mc:`mc tag set` command sets one or more tags to a bucket or object.

.. end-mc-tag-set-desc

An object can have up to 10 tags.

.. tab-set::

.. tab-item:: EXAMPLE
Expand Down
2 changes: 2 additions & 0 deletions source/reference/minio-mc/mc-tag.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ The :mc:`mc tag` command adds, removes, and lists tags associated to a bucket or

.. end-mc-tag-desc

An object can have up to 10 custom tags.

Subcommands
-----------

Expand Down
23 changes: 23 additions & 0 deletions source/reference/minio-server/settings/notifications/redis.rst
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,29 @@ Specify the password for the Redis server.

MinIO redacts this value when returned as part of :mc-cmd:`mc admin config get`.

User
~~~~

*Optional*

.. versionadded:: RELEASE.2024-03-21T23-13-43Z

.. tab-set::

.. tab-item:: Environment Variable
:sync: envvar

.. envvar:: MINIO_NOTIFY_REDIS_USER

.. tab-item:: Configuration Setting
:sync: config

.. mc-conf:: notify_redis user
:delimiter: " "


Specify the user for the Redis server.

Queue Directory
~~~~~~~~~~~~~~~

Expand Down