Divergence from aws s3: delete idempotency

[tonyipm]

Describe the bug
On aws s3, delete (rm) does not care whether the file to be deleted actually exists or not.
On versity, it does.

To Reproduce
Upload file 'x' to bucket 'titest', then delete twice:
AWS:

1. aws s3 cp x s3:https://tonyitest/x
   upload: ./x to s3:https://tonyitest/x

2. aws s3 rm s3:https://tonyitest/x
   delete: s3:https://tonyitest/x

3. aws s3 rm s3:https://tonyitest/x
   delete: s3:https://tonyitest/x

VERSITY GW:

1. vgaws s3 cp x s3:https://tonyitest/x
   upload: ./x to s3:https://tonyitest/x

2. vgaws s3 rm s3:https://tonyitest/x
   delete: s3:https://tonyitest/x

3. vgaws s3 rm s3:https://tonyitest/x
   delete failed: s3:https://tonyitest/x An error occurred (NoSuchKey) when calling the DeleteObject operation: The specified key does not exist.

Using wireshark, we can see Versity responds to the initial delete with HTTP 204, but the second one with 404:

80 389.891388505 127.0.0.1 → 127.0.0.1 HTTP 694 DELETE /tonyitest/x HTTP/1.1
81 389.891395435 127.0.0.1 → 127.0.0.1 TCP 66 7070 → 60850 [ACK] Seq=1 Ack=629 Win=133120 Len=0 TSval=4245578292 TSecr=4245578292
82 389.892798052 127.0.0.1 → 127.0.0.1 HTTP 168 HTTP/1.1 204 No Content

90 392.291199118 127.0.0.1 → 127.0.0.1 HTTP 694 DELETE /tonyitest/x HTTP/1.1
91 392.291206118 127.0.0.1 → 127.0.0.1 TCP 66 7070 → 60856 [ACK] Seq=1 Ack=629 Win=133120 Len=0 TSval=4245580692 TSecr=4245580692
92 392.291709834 127.0.0.1 → 127.0.0.1 HTTP/XML 418 HTTP/1.1 404 Not Found

AWS responds with 204 for both (and any further) deletion attempts.

Expected behavior
Versity to behave the same as aws.

Server Version
DEV03 [/WS/VGW.08.08.24/versitygw-main] :>./versitygw --version
Version :
Build : release-rpm
BuildTime: 2024-08-08_02:05:27PM

DEV03 [/WS/VGW.08.08.24/versitygw-main] :>uname -a
Linux ip-10-100-0-246 4.18.0-513.18.1.el8_9.x86_64 #1 SMP Thu Feb 1 03:51:05 EST 2024 x86_64 x86_64 x86_64 GNU/Linux

Additional context
aws-cli/1.24.10 Python/3.6.8 Linux/4.18.0-513.18.1.el8_9.x86_64 botocore/1.26.10

[benmcclelland]

I somehow have always missed that this would return success on AWS. I wonder how many buckets have items people thought they deleted but are still around due to some typo. Easy enough fix, PR opened.