To enhance privacy, reduce IP retention from 1 year to 6 months

[eobrain]

The current default retention period for IP addresses is one year. That is not good for privacy.

Defaults matter -- probably most admins do not change them.

Changing the default to two days will improve privacy while still allowing rate-limiting to guard against spam or abuse, and allow allowing production debugging of recent events.

See discussion in #6474 and in Reddit thread.

See also the matching documentation PR mastodon/documentation#1133 which probably should be merged if this PR is merged.

[rinsuki]

I think you don't need to modify SESSION_RETENTION_PERIOD, since even SESSION_RETENTION_PERIOD is longer than IP_RETENTION_PERIOD, session's IP address will be removed.

mastodon/app/workers/scheduler/ip_cleanup_scheduler.rb

Lines 19 to 20 in 2bad619

	SessionActivation.where('updated_at < ?', SESSION_RETENTION_PERIOD.ago).in_batches.destroy_all
	SessionActivation.where('updated_at < ?', IP_RETENTION_PERIOD.ago).in_batches.update_all(ip: nil)

btw, I think it would be good to ask option to server administrator in mastodon:setup task instead of change .env.production.sample.

[ineffyble]

Lowering SESSION_RETENTION_PERIOD to 2 days would have a user impact, since it'd mean users have to log back in every 2 days, right? In which case there's a tradeoff between convenience and privacy.

[eobrain]

Thanks for the review. Please take another look.

I changed the pull-request to only change the IP_RETENTION_PERIOD

Because as @rinsuki pointed out, changing just the IP retention period is enough to also remove the IP addresses from the session records.

And as @ineffyble pointed out, changing the
session retention period would cause a degradation of the user
experience.

[eobrain]

Also created a corresponding documentation PR: mastodon/documentation#1133

[smiba]

I don't really get this change. Yes, it would improve privacy, but it would also harm how one could moderate their instance.

IP banning, although easy to circumvent, is still an important part of moderation. Having longer retention would improve moderators and administrators ability to recognize hostile subnets or misbehaving / multi-account carrying IPs.

If an instance is already responsible for securing and hosting all of a user's social connections, including mention-only and followers-only posts, I don't see how an IP address is suddenly a noteworthy privacy risk.

I'm glad the session expiry is not affected (any more) in the commits because that would honestly absolutely damage user experience, but getting rid of IP addresses so fast would still affect moderators.

[eobrain]

TL;DR -- It seems moderators could still use IP Banning even with a short IP retention

Hi @smiba, thanks for bringing up the issue of use of IP addresses for moderation.

According to Blocking by IP in the Mastodon docs, the way that IP banning is implemented is not in the Mastodon software itself but in the firewall.

If that is true then once the IP address has been added to the firewall config it no longer needs to be stored by the Mastodon server.

So to allow an IP banning moderation process the Mastodon server only needs to store the IP long enough for the moderators to realize a user needs to be banned so they can use the IP address in the firewall config. After that it is OK if the IP address is deleted from Mastodon, as it will continue to be stored in the firewall config. Maybe two days is not enough time to realize a user needs to be banned, but the current default of one year seems too long.

Even if two days is too short, I still think this is a good default. The admins of an instance where moderators are doing IP banning can still change the default retention period to something more suitable to the speed of their moderation process.

(Maybe I misunderstand how IP banning is used in moderation. Please let me know of anything I got wrong above.)

[seano-vs]

I'm a privacy advocate, but this signal is just too valuable for moderation and this would cripple it. People create spam accounts from the same IPs all the time, and it's often spread out over months. At the very most, IPs should be one-way-hashed as UUIDs to mask the actual value- not expired after two days.

This is an odd change since this would only affect visibility of sensitive attributes only available to mods/admins. Like, they already have access to your everything- if they actually want your IP (and they will, for spam purposes) there are a hundred other ways to get it. I think not trusting the instance you host your account on is an issue outside the scope of this

[eobrain]

I defer to those with experience in moderation and accept that two days is too short a default retention time.

However is the current value of one year correct? Can we change it to something like two months?

The GDPR says that retention be “no longer than is necessary”, and we should try to minimize legal risk to administrators by having a default retention time that is legally defensible according to this standard.

Is there some retention period that would be the minimum to counter, say, 99% of spam, based on the experience of real-world administration of Mastodon?

[nemobis]

The MediaWiki CheckUser extension records the IP address of registered users for 3 months by default, so that would be one reasonable choice.
https://www.mediawiki.org/wiki/Extension:CheckUser

[smiba]

Please know that the expiry logic is not in .env.enviroment.sample, changing this file changes nothing other then the sample file that is supposed to be filled with default values

I've created the following PR a while ago and I recommend continuing in that one, as it properly changes the files required #23071

Your PR is missing the actual code change in the sidekiq worker.

[eobrain]

Thank you @smiba, this PR has been open for so long that I forgot about your PR.

I agree your PR is more complete and that is the one that should be merged.

If there is pushback on the 31 days as being too short for spam protection purposes, you might consider increasing it to six months as I did in this PR. It's not ideal for privacy, but still better than the current one year.

[boehs]

At the very most, IPs should be one-way-hashed as UUIDs to mask the actual value

Incredibly trivial to rainbow table that

[mjankowski]

I agree your PR is more complete and that is the one that should be merged.

Closing this based on comment from PR author that the other PR is more appropriate. Will comment more over there.