Aggregate information on deleting objects (#1218)

Creates a new object management page to aggregate information on
deleting objects.
Links to the page from other docs throughout.

Intentionally leaves most delete information where it is in existing
docs.
    
Closes #741

source/administration/batch-framework-job-expire.rst

@@ -86,7 +86,7 @@ Expire Batch Job Reference
  - ``object`` - Applies only to objects which do **not** have a ``DeleteMarker`` as the current version.
  - ``deleted`` - Applies only to objects which **do** Have a ``DeleteMarker`` as the current version.
 
- See :ref:`minio-bucket-versioning-delete` for more complete documentation on ``DeleteMarker`` or delete operations in versioned buckets.
+ See :ref:`minio-object-delete` for more complete documentation on ``DeleteMarker`` or delete operations in versioned buckets.
 
  * - ``rules.[n].name``
  - *Optional*

source/administration/bucket-replication.rst

@@ -102,7 +102,7 @@ to avoid costs associated to the "lost" data.
 Replication of Delete Operations
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-MinIO supports replicating delete operations, where MinIO synchronizes
+MinIO supports replicating :ref:`delete <minio-object-delete>` operations, where MinIO synchronizes
 deleting specific object versions *and* new 
 :s3-docs:`delete markers <delete-marker-replication.html>`. Delete operation
 replication uses the same :ref:`replication process <minio-replication-process>`

source/administration/bucket-replication/enable-server-side-multi-site-bucket-replication.rst

@@ -74,7 +74,7 @@ Click to expand any of the following:
 
  MinIO supports customizing the replication configuration to enable or disable the following replication behaviors:
 
- - Replication of delete operations
+ - Replication of :ref:`delete operations <minio-object-delete>`
  - Replication of delete markers
  - Replication of existing objects
  - Replication of metadata-only changes
@@ -92,7 +92,7 @@ Click to expand any of the following:
 .. dropdown:: Replication of Delete Operations
  :icon: fold-down
 
- MinIO supports replicating delete operations onto the target bucket. 
+ MinIO supports replicating :ref:`delete operations <minio-object-delete>` onto the target bucket. 
  Specifically, MinIO can replicate versioning :s3-docs:`Delete Markers <versioning-workflows.html>` and the deletion of specific versioned objects:
 
  - For delete operations on an object, MinIO replication also creates the delete marker on the target bucket.

source/administration/bucket-replication/enable-server-side-one-way-bucket-replication.rst

@@ -66,7 +66,7 @@ Click to expand any of the following:
 
  MinIO does *not* replicate delete operations resulting from the application of :ref:`lifecycle management expiration rules <minio-lifecycle-management-expiration>`.
 
- See :ref:`minio-replication-behavior-delete` for more complete documentation.
+ See :ref:`minio-replication-behavior-delete` and :ref:`minio-object-delete` for more complete documentation.
 
 .. dropdown:: Multi-Site Replication
  :icon: fold-down

source/administration/bucket-replication/enable-server-side-two-way-bucket-replication.rst

@@ -66,7 +66,7 @@ Considerations
 
  MinIO supports customizing the replication configuration to enable or disable the following replication behaviors:
 
- - Replication of delete operations
+ - Replication of :ref:`delete operations <minio-object-delete>`
  - Replication of delete markers
  - Replication of existing objects
  - Replication of metadata-only changes
@@ -97,7 +97,7 @@ Considerations
  MinIO does *not* replicate delete operations resulting from the application of :ref:`lifecycle management expiration rules <minio-lifecycle-management-expiration>`. 
  Configure matching expiration rules on both the source and destination bucket to ensure consistent application of object expiration.
 
- See :ref:`minio-replication-behavior-delete` for more complete documentation.
+ See :ref:`minio-replication-behavior-delete` and :ref:`minio-object-delete` for more complete documentation.
 
 .. dropdown:: Multi-Site Replication
  :icon: fold-down

source/administration/console/managing-objects.rst

@@ -46,7 +46,7 @@ Example actions the user may be able to perform include:
 - Manage tags
 - Inspect
 - Display versions
-- Delete
+- :ref:`Delete <minio-object-delete>`
 
 .. versionadded:: Console v0.24.0
 

source/administration/identity-access-management/policy-based-access-control.rst

@@ -605,7 +605,7 @@ Bucket Replication
 
  MinIO Extension for controlling API operations related to :ref:`Server-Side Bucket Replication <minio-bucket-replication-serverside>`.
 
- Required for synchronizing delete operations as part of MinIO server-side replication.
+ Required for synchronizing :ref:`delete operations <minio-object-delete>` as part of MinIO server-side replication.
 
  Supports the following additional :ref:`condition keys <minio-policy-conditions>`:
 

source/administration/object-management.rst

@@ -94,7 +94,7 @@ Object Organization and Planning
 --------------------------------
 
 Administrators typically control the creation and configuration of buckets.
-Client applications can then use :ref:`S3-compatible SDKs <minio-drivers>` to create, list, retrieve, and delete objects on the MinIO deployment.
+Client applications can then use :ref:`S3-compatible SDKs <minio-drivers>` to create, list, retrieve, and :ref:`delete <minio-object-delete>` objects on the MinIO deployment.
 Clients therefore drive the overall hierarchy of data within a given bucket or prefix, where Administrators can exercise control using :ref:`policies <minio-policy>` to grant or deny access to an action or resource.
 
 .. cond:: windows
@@ -129,7 +129,7 @@ Object Versioning
  :alt: Object with Multiple Versions
  :align: center
 
-The specific client behavior on write, list, get, or delete operations on a bucket depends on the versioning state of that bucket:
+The specific client behavior on write, list, get, or :ref:`delete <minio-object-delete>` operations on a bucket depends on the versioning state of that bucket:
 
 .. list-table::
  :stub-columns: 1
@@ -162,6 +162,8 @@ The specific client behavior on write, list, get, or delete operations on a buck
 
  Supports deleting any object version by version ID (hard delete).
  You cannot undo hard-delete operations.
+
+ Refer to :ref:`minio-object-delete` for more information.
  - Deletes the object
 
 See :ref:`minio-bucket-versioning` for more complete documentation.
@@ -200,7 +202,7 @@ Enabling bucket locking also enables :ref:`versioning <minio-bucket-versioning>`
 
 MinIO Object Locking provides key data retention compliance and meets SEC17a-4(f), FINRA 4511(C), and CFTC 1.31(c)-(d) requirements as per `Cohasset Associates <https://min.io/cohasset?ref=docs>`__.
 
-See :ref:`minio-object-locking` for more complete documentation.
+See :ref:`minio-object-locking` and :ref:`minio-object-delete` for more complete documentation.
 
 Object Lifecycle Management
 ---------------------------
@@ -224,5 +226,6 @@ See :ref:`minio-lifecycle-management` for more complete documentation.
 
  /administration/object-management/object-versioning
  /administration/object-management/object-retention
+ /administration/object-management/object-delete
  /administration/object-management/object-lifecycle-management
  /administration/object-management/data-compression

source/administration/object-management/object-delete.rst

@@ -0,0 +1,177 @@
+.. _minio-object-delete:
+
+===============
+Object Deletion
+===============
+
+.. default-domain:: minio
+
+.. contents:: Table of Contents
+ :local:
+ :depth: 2
+
+Overview
+--------
+
+This page summarizes how a ``DELETE`` operation affects objects depending on the configuration of the bucket that contains the object.
+
+Any combination of the following factors may impact how ``DELETE`` operations function:
+
+- :ref:`Bucket versioning <minio-bucket-versioning>`
+- :ref:`Object locking rules <minio-object-locking>`
+- :ref:`Object Lifecycle Management rules <minio-lifecycle-management>`
+- :ref:`Object tiering <minio-lifecycle-management-tiering>`
+- :ref:`Site <minio-site-replication-overview>` or :ref:`bucket <minio-replication-behavior-delete>` replication
+- :ref:`Scanner <minio-concepts-scanner>`
+
+Permissions
+-----------
+
+MinIO uses a :ref:`policy based access control <minio-policy>` system for access management.
+The user or service account must provide the correct policy action and conditions to allow a ``DELETE`` for the bucket and object.
+
+Unversioned Objects
+-------------------
+
+When performing a ``DELETE`` operation on an object in a bucket that does not have versioning enabled, the operation is straightforward.
+After verifying the user or service account has permission to perform the ``DELETE`` operation, MinIO permanently removes the object.
+
+The user or service account requesting the delete action the action must have the :policy-action:`s3:DeleteObject` action permission for the bucket and object.
+
+Versioned Objects
+-----------------
+
+``DELETE`` operations work differently when an object is versioned.
+
+The user or service account must have the :policy-action:`s3:DeleteObjectVersion` action permission for the bucket and object.
+
+Delete operations on the current version
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A ``DELETE`` operation on a versioned object that does not specify a version UUID results in the creation of a ``DeleteMarker`` placed as the ``head`` of the object.
+
+In this scenario, MinIO does not actually remove the object or any of its versions from the disk.
+All existing versions of the object remain available to access by specifying the version's UUID.
+When a ``DeleteMarker`` is the head for the object, MinIO does not serve the object for ``GET`` requests that do not specify a version ID.
+Instead, MinIO returns a ``404``-like response. 
+
+You can find the UUID of object versions with :mc-cmd:`mc ls --versions`.
+
+To remove the current version of the object from the drive, find the UUID of the version, and then use :mc-cmd:`mc rm --version-id=UUID ... <mc rm --version-id>` to delete the current version.
+In this scenario, the immediately preceding version of the object then becomes the current version of the object served for ``GET`` requests of the object with no UUID specified.
+
+.. warning::
+
+ Specifying a ``version-id`` in a DELETE operation is irreversible.
+ MinIO removes the specified version from the drive and **cannot** retrieve it.
+
+Delete operations on a prior version
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To delete prior versions of an object, specify the version's UUID.
+You can retrieve the version UUID with :mc-cmd:`mc ls --versions`. 
+When the ``DELETE`` request specifies a ``version-id`` and the user has the correct permissions to delete the object version`, MinIO permanently removes the specified version from the drive.
+
+.. warning::
+
+ Specifying a ``version-id`` in a DELETE operation is irreversible.
+ MinIO removes the specified version from the drive and **cannot** retrieve it.
+
+Delete all versions
+~~~~~~~~~~~~~~~~~~~
+
+Use :mc-cmd:`mc rm --versions` to delete *all* versions of an object.
+This is irreversible.
+
+Lifecycle Management Expiration
+-------------------------------
+
+You can define one or more :ref:`lifecycle management expiration rule(s) <minio-lifecycle-management-create-expiry-rule>` to expire objects after a certain version number count or a certain period of time.
+When more versions exist than the rule specifies, or when a version is older than specified, MinIO permanently removes the object version from the drive.
+
+These rules rely on the :ref:`scanner <minio-concepts-scanner>` to process the rule on the bucket.
+The scanner operates as a lower priority continuous process where ``READ`` and ``WRITE`` actions are preferred.
+Because of this, object versions that meet the requirements for expiration may not immediately be removed from MinIO.
+
+See the :ref:`scanner <minio-concepts-scanner>` page for more details on how the scanner works and configuration options.
+
+``DeleteMarkers`` are their own objects.
+Lifecycle rules can remove ``DeleteMarkers`` that are the only remaining versions of their objects.
+
+Retained Objects
+----------------
+
+MinIO protects objects subject to a :ref:`locking rule <minio-object-locking>` from being overwritten or deleted.
+These rules require that objects be retained until either the rule expires or is removed.
+
+``DELETE`` operations on locked objects without a specified version result in the creation of a `DeleteMarker` for the object.
+However, the object versions themselves are retained as required by the lock.
+
+``DELETE`` operations that specify an object version are subject to the retention rules.
+MinIO protects object versions subject to a lock from being overwritten or deleted until the lock expires or is removed.
+
+Replicated Objects
+------------------
+
+Replication duplicates objects from one location to another.
+MinIO supports replication at the bucket level or the cluster ("site") level.
+
+Delete operations may or may not replicate, depending on the type of replication and how the replication is configured.
+
+Site Replication
+~~~~~~~~~~~~~~~~
+
+For clusters with :ref:`multi-site replication <minio-site-replication-overview>` enabled, MinIO replicates all ``delete`` operations performed on any cluster to each of the other clusters in the peer group.
+
+Delete behavior on any single peer follows the same processes as any MinIO deployment.
+
+Bucket Replication
+~~~~~~~~~~~~~~~~~~
+
+With :ref:`bucket replication <minio-bucket-replication>`, MinIO supports replicating delete operations between a source bucket and a configured remote bucket.
+MinIO synchronizes deleting specific object versions *and* new :s3-docs:`delete markers <delete-marker-replication.html>`. 
+Delete operation replication uses the same :ref:`replication process <minio-replication-process>` as all other replication operations. 
+
+MinIO requires *explicitly enabling* versioned deletes and delete marker replication. 
+Use the :mc-cmd:`mc replicate add --replicate` field to specify either ``delete`` and ``delete-marker`` or both to enable versioned deletes and delete marker replication, respectively. 
+To enable both, specify both strings using a comma separator: ``delete,delete-marker``.
+
+For delete marker replication, MinIO begins the replication process after a delete operation creates the delete marker. 
+MinIO uses the ``X-Minio-Replication-DeleteMarker-Status`` metadata field for tracking delete marker replication status. 
+In :ref:`active-active <minio-bucket-replication-serverside-twoway>` replication configurations, MinIO may produce duplicate delete markers if both clusters concurrently create a delete marker for an object *or* if one or both clusters were down before the replication event synchronized.
+
+For replicating the deletion of a specific object version, MinIO marks the object version as ``PENDING`` until replication completes. 
+Once the remote target deletes that object version, MinIO deletes the object version on the source.
+While this process ensures near-synchronized version deletion, it may result in listing operations returning the object version after the initial delete operation. 
+MinIO uses the ``X-Minio-Replication-Delete-Status`` for tracking delete version replication status.
+
+MinIO only replicates explicit client-driven delete operations. 
+MinIO does *not* replicate objects deleted by :ref:`lifecycle management expiration rules <minio-lifecycle-management-expiration>`. 
+For :ref:`active-active <minio-bucket-replication-serverside-twoway>` configurations, set the same expiration rules on *all* of of the replication buckets to ensure consistent application of object expiration.
+
+.. admonition:: MinIO Trims Empty Object Prefixes on Source and Remote Bucket
+ :class: note, dropdown
+
+ If a delete operation removes the last object in a bucket prefix, MinIO recursively removes each empty part of the prefix up to the bucket root.
+ MinIO only applies the recursive removal to prefixes created *implicitly* as part of object write operations.
+ MinIO does not recursively remove prefixes created using an explicit directory creation command, such as :mc:`mc mb`.
+
+ If a replication rule enables replication delete operations, the replication process *also* applies the implicit prefix trimming behavior on the destination MinIO cluster.
+
+ For example, consider a bucket ``photos`` with the following object prefixes:
+
+ - ``photos/2021/january/myphoto.jpg`` // ``2021/january/`` created implicitly based on the object name
+ - ``photos/2021/february/myotherphoto.jpg`` // ``2021/february/`` created implicitly based on the object name
+ - ``photos/NYE21/NewYears.jpg`` // ``NYE21/`` explicitly created in the bucket
+
+ ``photos/NYE21`` is the *only* prefix explicitly created using :mc:`mc mb`.
+ All other prefixes were *implicitly* created as part of writing the object located at that prefix. 
+
+ - A command removes ``myphoto.jpg``. 
+ MinIO automatically trims the empty ``/january/`` prefix. 
+
+ - A command then removes the ``myotherphoto.jpg``. 
+ MinIO automatically trims the ``/february/`` prefix *and* the now-empty ``/2021`` prefix. 
+
+ - A command removes the ``NewYears.jpg`` object. 
+ MinIO leaves the ``/NYE21/`` prefix remains in place since it was *explicitly* created.

source/administration/object-management/object-retention.rst

@@ -59,7 +59,7 @@ SEC17a-4(f), FINRA 4511(C), and CFTC 1.31(c)-(d) requirements as per
  :align: center
  :width: 600px
 
- Delete operations follow normal behavior in 
+ :ref:`Delete operations <minio-object-delete>` follow normal behavior in 
  :ref:`versioned buckets <minio-bucket-versioning-delete>`, where MinIO
  creates a ``DeleteMarker`` for the object. However, non-Delete Marker 
  versions of the object remain under the retention rules and are protected 
@@ -72,7 +72,7 @@ SEC17a-4(f), FINRA 4511(C), and CFTC 1.31(c)-(d) requirements as per
  :align: center
  :width: 600px
 
- MinIO blocks any attempt to delete a specific object version held under
+ MinIO blocks any attempt to :ref:`delete <minio-object-delete>` a specific object version held under
  WORM lock. The earliest possible time after which a client may delete
  the version is when the lock expires.
 

source/reference/deprecated/mc-ilm-add.rst

@@ -130,7 +130,7 @@ Parameters
 
  - :mc-cmd:`~mc ilm add --expire-delete-marker`
 
- For more complete documentation on object expiration, see :ref:`minio-lifecycle-management-expiration`.
+ For more complete documentation on object expiration, see :ref:`minio-lifecycle-management-expiration` and :ref:`minio-object-delete`.
 
 .. mc-cmd:: --expire-delete-marker
  :optional:
@@ -147,8 +147,7 @@ Parameters
  Slow scanning due to high IO workloads or limited system resources may delay application of lifecycle management rules. 
  See :ref:`minio-lifecycle-management-scanner` for more information.
 
- For more complete documentation on object expiration, see
- :ref:`minio-lifecycle-management-expiration`.
+ For more complete documentation on object expiration, see :ref:`minio-lifecycle-management-expiration` and :ref:`minio-object-delete`.
 
 .. mc-cmd:: --transition-days
  :optional:

source/reference/deprecated/mc-ilm-edit.rst

@@ -146,6 +146,8 @@ Parameters
 
  - :mc-cmd:`~mc ilm edit --expired-object-delete-marker`
 
+ See :ref:`minio-object-delete` for more information.
+
 .. mc-cmd:: --expired-object-delete-marker
  :optional:
 
@@ -161,7 +163,7 @@ Parameters
  MinIO uses a :ref:`scanner process <minio-concepts-scanner>` to check objects against all configured
  lifecycle management rules. Slow scanning due to high IO workloads or
  limited system resources may delay application of lifecycle management
- rules. See :ref:`minio-lifecycle-management-scanner` for more information.
+ rules. See :ref:`minio-lifecycle-management-scanner` and :ref:`minio-object-delete` for more information.
 
 .. mc-cmd:: --noncurrentversion-expiration-days
  :optional:

source/reference/minio-mc/mc-ilm-rule-add.rst

@@ -144,7 +144,7 @@ Parameters
 
  - :mc-cmd:`~mc ilm rule add --expire-delete-marker`
 
- For more complete documentation on object expiration, see :ref:`minio-lifecycle-management-expiration`.
+ For more complete documentation on object expiration, see :ref:`minio-lifecycle-management-expiration` and :ref:`minio-object-delete`.
 
 .. mc-cmd:: --expire-delete-marker
  :optional:
@@ -161,8 +161,7 @@ Parameters
  Slow scanning due to high IO workloads or limited system resources may delay application of lifecycle management rules. 
  See :ref:`minio-lifecycle-management-scanner` for more information.
 
- For more complete documentation on object expiration, see
- :ref:`minio-lifecycle-management-expiration`.
+ For more complete documentation on object expiration, see :ref:`minio-lifecycle-management-expiration` and :ref:`minio-object-delete`.
 
 .. mc-cmd:: --transition-days
  :optional: