For GCE deployments, automatically set max_mss, or prompt the user to set it

[pjl]

Prior to #976, max_mss used to be set for GCE deployments automatically (see https://github.com/trailofbits/algo/pull/976/files#diff-28f2b95533afb47cbec1d823b0f1a941L535). Now that max_mss is undefined, GCE deployments no longer appear to work out of the box. I brought up a new GCE deployment on November 23, and noticed I was unable to load sites in my browser until I redeployed with max_mss set to 1316, the default value listed in config.cfg.

Can the previous behavior of automatically setting max_mss for GCE deployments be brought back, or can the user be prompted to set max_mss (maybe default to 1316)?

If either approach sounds good, I can open a PR with the changes.

Thanks!

[woodruffw]

I was able to confirm this -- many requests (especially ones to google properties, interestingly) will fail with the default MSS/MTU on GCE.

Can the previous behavior of automatically setting max_mss for GCE deployments be brought back

This sounds good to me. We'd appreciate a PR!

[davidemyers]

It might be worthwhile to handle WireGuard separately since it allows the MTU to be set directly in the client configuration files.

For example, create an additional variable wireguard_mtu which is only added to the WireGuard client config files if set. You'd also need to change Algo's firewall rules so that MSS clamping does not apply to the WireGuard client subnet.

The default MTU for WireGuard set by wg-quick is 1420. I have a non-Algo GCE VPS which has an MTU of 1460, so maybe the correct MTU for WireGuard with Algo on GCE is 1380.

[davidemyers]

Rather than add a second variable for WireGuard as I suggested above and force users to do MTU and MSS math, I've created a fork that replaces max_mss with a new variable reduce_mtu. If reduce_mtu is non-zero the proper MSS values for IPsec and MTU value to WireGuard will be calculated automatically. For GCE reduce_mtu: 40 should work.

However I'm not currently set up to deploy to GCE. Could a GCE user following this issue give it a try? Get a copy of my fork with:

git clone -b reduce_mtu https://github.com/davidemyers/algo.git

Then edit config.cfg and set reduce_mtu: 40. This will set the IPv4 MSS to 1320 instead of 1316, but 1320 should work (I hope).

If you find this works for you I'll submit a PR.

@lperry This code doesn't set reduce_mtu by default for GCE as I'm not sure the proper place to do that.

[jackivanov]

@davidemyers I think it might be confusing. How about just put fixed mtu size in the config? I'll file a PR with the changes for GCE shortly

[davidemyers]

@jackivanov I was trying to come up with a single variable which would work for both IPsec and WireGuard which have different MTUs.

[pjl]

@davidemyers I'll try to give it a shot tomorrow or day after. Thanks!

[davidemyers]

@jackivanov How about if I simplify the description:

# Reduce the MTU of the VPN tunnel
# Some cloud and internet providers use a smaller MTU (Maximum Transmission
# Unit) than the normal value of 1500 and if you don't reduce the MTU of your
# VPN tunnel some network connections will hang. For example, a Google GCE
# server has an MTU of 1460, so set 'reduce_mtu' to 40 (1500 - 1460 = 40).
reduce_mtu: 0

[pjl]

@davidemyers Your branch looks good to me. I brought up a new GCE deployment with your changes, and verified I was able to browse a few sites with either IPsec or WireGuard active.

[jackivanov]

@davidemyers I'd rather let users chose the MTU size, instead of hardcoded values. I'll send a branch with examples later today

[jackivanov]

@davidemyers it's here

[davidemyers]

@jackivanov I have some questions about that approach.

Apple devices use an MTU of 1400 for IPsec, and the Linux and iOS WireGuard implementations use an MTU of 1420. These seem to be working fine for Algo users on cloud providers with a normal MTU.

But when the MTU must be reduced, how does the user set an optimal value for both IPsec and WireGuard?

Also, for IPv6 isn't MSS = MTU - 60?

[jackivanov]

@davidemyers I thought that we can reduce this complexity by setting the same value for both IPsec and WG. Would there be any problems if we do that? I haven't encountered any problems so far using my approach, but if there would be any in the future we'd better to use reduce_mtu
Yes, 60 for IPv6.

[davidemyers]

Aside from being slightly non-optimal for WireGuard, the only problems I can think of are:

1. If someone uses ping to determine their MTU over WireGuard, their connections over IPsec could still hang.

2. People like me might point out that it's slightly non-optimal for WireGuard. 😃

[jackivanov]

OK, I see. Let's stick with reduce_mtu then. Could you send a PR please?

[davidemyers]

Will do. This change will make the MTU instructions in the troubleshooting document obsolete. Is it OK with you if I submit a PR without updating troubleshooting and go back and update it in a later PR?

[jackivanov]

@davidemyers I think it would be better to put everything in one PR

[davidemyers]

OK, then I will need a day or so to put the troubleshooting section together, then I'll submit a PR.

[davidemyers]

@jackivanov I finished the PR, but then had another idea.

What if by default reduce_mtu is set is based on ansible_default_ipv4['mtu']? If the user hasn't set reduce_mtu in config.cfg and ansible_default_ipv4['mtu'] < 1500, then set reduce_mtu to 1500 - ansible_default_ipv4['mtu'].

This should work for GCE and perhaps other providers. If you think this is a good idea I'll add it to the PR. Of course it won't handle the case where the MTU is limited on the client side.

Edited to add: I went ahead and implemented this and it works when deploying to GCE. When deploying to DigitalOcean it makes no MTU changes unless reduce_mtu is changed in config.cfg.

[jackivanov]

@davidemyers Yes, I like this idea.

[davidemyers]

@lperry @woodruffw I submitted PR #1253 which should allow you to deploy to GCE without any MTU or MSS changes on your part. Please give it a try.