Exclude certain pages from being tracked

[metmarkosaric]

"Is there an option to exclude certain resources so that these are not recorded analytic events? For example, if there is a need to preclude a list of pages, can these be added to Plausible as non-interactive/non-recording resources? The script is currently embedded dynamically. It would require additional development to exclude it from the build and deploy process."

Update: This is now live! Instructions: https://plausible.io/docs/excluding-pages

[Vigasaurus]

I'm working on this and #112 together, is there any preference on what we send back for events that are excluded (either by IP or page path)? Just still send 202 and quietly not count the view, or send back a code to actually indicate the lack of counting (maybe a 403)? I figure the former is better from a "protecting site settings standpoint", but the latter may be more useful for debugging/if people forget an exclusion is set up.

[ukutaht]

@Vigasaurus I think 202 is fine for this. From an outside/public point of view, the /api/event endpoint just accepts the pageview for processing. The return code is not really meant to indicate anything other than 'is this a valid request'. We don't return anything special if the pageview is ignored due to a bot user-agent, for example.

I would like to discuss how you're planning to implement these two features. For pageview processing, performance is critical and we definitely cannot look anything up from PostgreSQL when an event comes in. Are you building an in-memory cache for IP and page exclusions? That's the only way I can see it being implemented.

[Vigasaurus]

Hmm, I wasn't planning on doing anything in-memory - don't you think that'd get too large too quickly?

If we were gonna go the in-memory route, do you have any preference for tool? Otherwise, what if we did something where we let the event get creates in the same current speed, and then after the processing/adding to clickhouse is done we run through to check if the IP/page should have been excluded and remove the recently created event then? Or alternatively, we could try saving all events regardless and then only use the ones not excluded when sending data back to clients?

[Vigasaurus]

Wait also I may have misunderstood - do you mean that doing Postgres with a in-memory cache would be the right way? i.e. only hit Postgres the first time per exclusion hit per server start? If that's what you meant then I think that works perfect, and I can do that no problem. I thought you meant that it should be in-memory only, originally.

[ukutaht]

what if we did something where we let the event get creates in the same current speed, and then after the processing/adding to clickhouse is done we run through to check if the IP/page should have been excluded and remove the recently created event then?

This might work for pages but not for IP addresses since we don't persist the IP address at all. The only time the IP address is accessible is during the http request to /api/event.

do you mean that doing Postgres with a in-memory cache would be the right way?

Yes, although I would load the whole thing in memory during server boot rather than lazy-loading as pageviews come in. I think https://github.com/whitfin/cachex/ should work well for this purpose. The proper cache record would have to be invalidated when a user adds an exclusion rule.

Alternative approaches

To zoom out a little bit, I'd prefer people to not send event to Plausible at all if they don't want certain things to be tracked. That way we don't have to absorb garbage traffic and maintain these in-memory caches just to keep track of what to throw away.

These exclusions would be best implemented in the browser before these events ever reach our servers. For example, we could support a script attribute like <script async defer data-exclude="/admin"</scrip> which would take any regex and not send a pageview if the current page matches it.

That combined with more powerful filtering on the dashboard should be enough for page exclusions IMO. Either a static config in your script embed or completely dynamic as you're querying your stats.

I've also thought about implementing a simpler way to exclude your own views while testing. We could instruct people to just run localStorage.plausible_ignore=true in their console and then the script would not send a pageview based on the localStorage contents. It's still likely that people will ask for IP-based exclusion though, I don't think we can get around that.

[Vigasaurus]

Those are all great points, I actually do really like the concept of using something client-side to prevent unnecessary garbage data clogging server time. I think the biggest things people want IP exclusions for are for preventing their own test data - I think the local storage flag as you said would make it easy for people to exclude that, and we would just need to make some sort of easy way/guide for people to set said flag on the specific domains.

I'll look into it and see what I can come up with. I 100% think client-side prevention can be both cleaner and easier to administer.

[codeofsumit]

Just to add to this - we have pages like "loggin-in" that shows up after authentication redirects etc. Those are pages that are "most visited" for our web-app but we would like to exclude it from the stats. I have no preference if I should block it client-side or inside the dashboard
https://plausible.io/tresor.one

[Kansuler]

I'd like exclude urls based on pattern/regex or similar. In my use case we generate secret links with expiration time, and it's something we don't want to record in our statistics. Possibly aggregate in some way to exclude the secret parameter. example: /contract/b14f8414-71c9-441d-a6ab-ad7f12525309. Either exclude it entierly, or aggregate unique links to /contract/<id>

[metmarkosaric]

This is now live! Instructions: https://plausible.io/docs/excluding-pages

[that-guy-iain]

Since it's kinda the same theme. I'm wanting to ignore either a specific user-agent or if it has some get query such as source. Basically, I have monitoring that includes a full-page load via chrome and I would rather these checks weren't included.

[Vigasaurus]

If you're able to run javascript in your monitoring software ahead of its load, or are able to add lines in your html based on an environment variable or similar, you could use the local storage flag to prevent the monitoring tool from sending events to plausible; see: https://plausible.io/docs/excluding

i.e. something akin to

{process.env.MONITOR && <script>window.localStorage.plausible_ignore=true</script>}
<script src="...plausible.js" data-domain="..."></script>

Of course the example only makes sense for specific tech stacks, but I hope that makes sense. You could also similarly simply exclude the plausible script line based on the same environment variable-style setup.

[metmarkosaric]

We now have an easy and simple way to exclude certain pages or sections from being tracked directly in the site settings https://plausible.io/docs/top-pages#block-traffic-from-specific-pages-or-sections