Document that S3 ACLs must be enabled #952
Conversation
|
I'm not sure there's any good reason for Mastodon to require Object-level ACLs, maybe kreeti/kt-paperclip#92 would be a better approach instead? |
|
One place that Mastodon explicitly sets ACLs is when suspending an account. All media from the suspended account is made private using ACLs. There is a PR (mastodon/mastodon#17979) that adds support disabling the use of ACLs in Mastodon. However a comment from @Gargron suggests that hiding media from suspended accounts is an important feature. Besides the places where Mastodon explicitly sets ACLs, I needed to enable ACLs to successfully attach a picture to a toot. I tested this using Mastodon v3.5.3 against AWS S3 and GCS. |
|
That's a great point, I wasn't aware of that feature.
…On Sat, Jul 2, 2022 at 8:17 PM Austin Wise ***@***.***> wrote:
One place that Mastodon explicitly sets ACLs is when suspending an
account. All media from the suspended account is made private using ACLs
<https://github.com/mastodon/mastodon/blob/a233a9bfb5f384e89bdaef6e519fa20db2a99ae5/app/services/suspend_account_service.rb#L80>.
There is a PR (mastodon/mastodon#17979
<mastodon/mastodon#17979>) that adds support
disabling the use of ACLs in Mastodon. However a comment from @Gargron
<mastodon/mastodon#17979 (comment)>
suggests that hiding media from suspended accounts is an important feature.
Besides the places where Mastodon explicitly sets ACLs, I needed to enable
ACLs to successfully attach a picture to a toot. I tested this using
Mastodon v3.5.3 against AWS S3 and GCS.
—
Reply to this email directly, view it on GitHub
<#952 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABZCV6A2MCICW6XGORTRZ3VSDSZDANCNFSM52PQNS5Q>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
This actually took down my instance. I enabled S3 uploads but used the recommended configuration from Amazon - i.e. no ACLs. This meant that all the uploads fail because Amazon does not have a "transparently ignore public ACLs but let the uploads succeed" option. I didn't notice that jobs were failing at the time for two reasons:
This resulted in a massive queue of Sidekiq jobs piling up and Sidekiq constantly using more RAM than I had. Previously I had added swap space specifically for occasional memory excursions, but constant usage burns through EBS's burst balance really quick. And once you run out of EBS burst the instance locks up and dies. |
|
This just bit me. Can this one line PR please be reviewed and merged? |
AustinWise
force-pushed
the
austin/ObjectStorageSetup
branch
from
14e0057
to
048e0b7
Compare
|
I rebased the commit to resolve merge conflicts and update the description of this PR with more details. |
AustinWise
force-pushed
the
austin/ObjectStorageSetup
branch
from
048e0b7
to
702050f
Compare
|
@nightpool Can you take another look at this PR. I updated the description with a more detailed description on why people are likely to incorrectly setup their S3-compatible storage. And there have been a couple of commenters mentioning that they hit this issue. |
|
@AustinWise I don't have permission to merge this PR but it LGTM |
When using the web GUI to create a bucket, both AWS S3 and GCS have ACLs disabled by default.
AustinWise
force-pushed
the
austin/ObjectStorageSetup
branch
from
702050f
to
0c70e68
Compare
|
Although note that the underlying bug here was fixed in 4.1.0 hopefully so
it's still possible to use non-ACL services, it's just highly recommended
to have ACLs enabled
…On Sat, Jul 2, 2022 at 3:44 PM Austin Wise ***@***.***> wrote:
When using the web GUI to create a bucket, both AWS S3 and Google Cloud
Storage have object-level ACLs disabled by default.
It appears I'm not the only person who has had trouble with this:
- mastodon/mastodon#17435
<mastodon/mastodon#17435>
- mastodon/mastodon#17978
<mastodon/mastodon#17978>
------------------------------
You can view, comment on, or merge this pull request online at:
#952
Commit Summary
- 14e0057
<14e0057>
Document that S3 ACLs must be enabled
File Changes
(1 file <https://github.com/mastodon/documentation/pull/952/files>)
- *M* content/en/admin/config.md
<https://github.com/mastodon/documentation/pull/952/files#diff-38e8ddf43a9a32293552d94137f678df4008dd396d3859b9908c21eb0557faf1>
(2)
Patch Links:
- https://github.com/mastodon/documentation/pull/952.patch
- https://github.com/mastodon/documentation/pull/952.diff
—
Reply to this email directly, view it on GitHub
<#952>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABZCV7JPUQ6O7EE66CCAZTVSCL2HANCNFSM52PQNS5Q>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
When using S3 compatible object stores to save attachments, object-level ACLs must be enabled. If they are not enabled, attaching pictures to toots will fail. For some object stores, the error message does not indicate the root cause of the problem.
Unfortunately it is likely that people will accidentally disable ACLs when first setting up Mastodon. This is because both AWS S3 and Google cloud storage disable object-level ACLs by default. The GUIs for creating buckets recommend somewhat strongly that they should not be enabled.
For example, the AWS docs say
And the creation UI looks like this:
It appears I'm not the only person who has had trouble with this: