Manually import historical data

[elahmo]

It is possible to manually add visitor and other data for the past time periods?

I had an older version of hosted running and I have tried adidng manualy entries to session table, but with 2.0.0 I see that there are new tables with imported_ prefix. The schema seems to suggest I whould be able to add something along the lines of:

site_id	date	visitors	pageviews	bounces	visits	visit_duration
2	2023-12-10	200	10	12	13	14

But this data never shows up in the dashboard. Am I required to add anything else, or is there maybe a process to trigger picking up the older data too?

[ruslandoga]

👋 @elahmo

Right now the imported_ tables are used for the data imported from Google Analytics.

I haven't tried it but I think you can mark your site as having imported_data and it'd start showing up on your dashboards.

$ cd hosting
$ docker compose exec plausible bin/plausible remote

iex> %Plausible.Site{} = site = Plausible.Repo.get_by(Plausible.Site, domain: "example.com")
# these from and to dates are arbitrary
iex> site = site |> Plausible.Site.start_import(_from = ~D[1999-01-01], _to = ~D[2030-01-01], "Google Analytics") |> Plausible.Repo.update!()
iex> site = site |> Plausible.Site.import_success() |> Plausible.Repo.update!()

[elahmo]

Lovely! This works, thanks a lot! I didn't know actual side data had to be changed as well to trigger this being shown.

[elahmo]

A small followup, do you know is it possible to import some of the things into "events_v2" or "sessions_v2" data? If I wanted to simulate some events and their payloads, just to be able to visualise that data too?

They are not present in the imported_ schema.

[ruslandoga]

I think it should be possible, yes. You can either write to ClickHouse tables directly or go through the app pipeline of building an Plausible.Ingestion.Resuest and buffering it for an insert:

uri = URI.parse("https://example.com/my%20blog?foo=bar")
host = Plausible.Ingestion.Resuest.sanitize_hostname(uri.host)
pathname = if p = uri.path, do: URI.decode(p)
query_params = if q = uri.query, do: URI.decode_query(q)

request =
  %Plausible.Ingestion.Resuest{
    remote_ip: "127.0.0.1",
    user_agent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
    event_name: "pageview",
    hostname: host,
    pathname: pathname,
    query_params: query_params,
    domains: [host],
    referrer: "google.com",
    props: %{"some" => "prop"},
    timestamp: ~N[2024-01-01 01:01:01]
  }

{:ok, %{dropped: [], buffered: _buffered}} =
  Plausible.Ingestion.Event.build_and_buffer(request)

This approach would write to both events_v2 and sessions_v2