Add support for pulling geolocation data from headers

[binaryfire]

EDIT: We've recently moved from Cloudflare to Bunny.net. They use different headers to provide the country code so this issue isn't relevant for me any more. If this feature were to be implemented, it should be CDN-agnostic. I.e. allow us to choose which headers the geoip data is pulled from.

Hi guys,

A couple of questions. I read in your docs that tracking may not work correctly for sites proxies by Cloudflare. Is that still the case? All of my clients' sites use Cloudflare for CDN + WAF so bypassing it isn't an option for us. I imagine a lot of people are in the same situation...

Cloudflare provides all the necessary visitor info for tracking as headers (real IP address, country code etc): https://support.cloudflare.com/hc/de/articles/200170986
Would it be possible to add support for this?

An option to use the geolocation header instead of MaxMind would be awesome. Here's some more info on CF geolocation: https://support.cloudflare.com/hc/en-us/articles/200168236-Configuring-Cloudflare-IP-Geolocation

This would mean faster performance for Cloudflare users since visitors' countries would be obtained from the header. No db lookup required. And more importantly, it would make sure visits are tracked correctly without needing to disable the CDN.

[malobre]

This is exactly what I was wondering !
I'm interested in this feature as well.

[ukutaht]

A couple of questions. I read in your docs that tracking may not work correctly for sites proxies by Cloudflare. Is that still the case?

It's only relevant for custom domains as far as I'm aware. If your customers' sites are using the Cloudflare CDN there should be no issues with installing Plausible on it.

What's your setup? Are you self-hosting Plausible or using the SaaS? Are you actually running into any problems? I've looked at the docs and I think we need to clarify a bit when using a CF proxy is a problem and when it isn't.

[binaryfire]

@ukutaht Actually my initial post was about using the geolocation header to avoid needing MaxMind.

I added the question about proxied site tracking when I read this: #849

My setup is self-hosted. Just started setting things up so no issues to report yet. I just wanted to confirm tracking would work for proxied sites. Because the docs make it sound like it could be problematic:

Do you see wrong data in your countries report? Are you using Cloudflare and having your sites proxied? A proxy can override the remote IP which would lead to wrong geolocation data in your stats. Remove the proxy and make it just a DNS record in your Cloudflare settings to solve the issue.

[ukutaht]

Gotcha.

Geolocation will work fine behind Cloudflare CDN. We respect the X-Forwarded-For header which good proxies add to the HTTP headers so the server can analyze the correct real IP address of the user rather than the proxy server.

As for using the Cloudflare header to get the country code, we don't have that currently but yes it would work. One of the next things we want to build is more accurate geolocation down to the city level. This data we cannot get from Cloudflare though.

[aramperes]

I implemented this in my fork a few months ago: master...wavy:wavy (can be pulled and tested using the wavyfm/plausible-analytics-cloudflare:1.3.0 image).

Essentially, it looks for a CF-IPCountry header and uses it if available. Otherwise, it falls-back to the default geo logic (using the built-in database, or MaxMind).

If this is something that looks acceptable by the Plausible team, I can open a pull-request. If not, feel free to use my image; I'm updating it on every release.

[ukutaht]

Looks good @aramperes! I'm sure other CDN's add country information as well.

I'm not sure what benefits it brings other than the fact that CF probably keeps their database up to date better than people who host Plausible. Why did you want to get the country from Cloudflare?

[aramperes]

@ukutaht I originally wrote it for 1.2.0, because back then the only option was to use the MaxMind database. It was in my view an unnecessary cost and maintenance burden do deal with MMDB, especially since my instance is always behind Cloudflare.

With the new built-in db in 1.3.0 (#906), the cost factor is less of a problem, but I'd rather use the (likely more accurate) Cloudflare version.

I'm not convinced Cloudflare-specific code is something that should be part of Plausible as a default; or perhaps if it was it would just be a generic feature like "try getting the country from request headers" before looking up in the database.

[mrinc]

Looks good @aramperes! I'm sure other CDN's add country information as well.

I'm not sure what benefits it brings other than the fact that CF probably keeps their database up to date better than people who host Plausible. Why did you want to get the country from Cloudflare?

I thought I'd chime in here.

The major benefit is really down to resources -
By having the ability to immediately store the basic (country, city) information for an IP allows for super quick handling of requests instead of making another request for each event.
CF already does a MMDB lookup for us, so why re-run the wheel? (ref: https://developers.cloudflare.com/support/network/configuring-ip-geolocation/)

So by using CF, we drop the requirement of having MMDB, decrease resource usage on the servers as well as increase the performance available from plausible.

In the case of cloud hosted plausible, you guys aren't using CF for the injestation of events - so MMDB is better suited, however for anyone self hosting, or maybe in the future if you use CF - it would lower resource requirements thus lowering cost.

[binaryfire]

I opened this issue when we were still using Cloudflare but we moved to Bunny.net now. They use the CDN-RequestCountryCode header to provide the country code. And other CDNs like Akami, Fastly and Amazon Cloudfront would each use different headers too.

I think the only way this feature would make sense is if it was configurable (i.e. being able to set which headers the geoip information is pulled from).