feat: Add a way to retrieve dimensions from multiple cubes

[bprusinowski]

This PR adapts the application logic to support retrieving dimensions from multiple cubes at once.

GQL / SPARQL

• Consolidated several GQL queries together into DataCubesComponents that supports retrieving the components from several cubes at once
• Removed Components query (replaced by DataCubesComponents)
• Removed ComponentsWithHierarchies and DimensionHierarchy queries (hierarchies are now fetched by default in DataCubesComponents). As we generally use them in many different places across the app, it's more developer-friendly to have one source of truth (and also, it might be even more performant to have one query, instead of several different ones fired from different parts of the app). We can also see how the new version of query-cube-hierarchy library improves performance, as then it might be even less worrying, or we can bring back the old way of separating hierarchies fetching to the specific places that use them
• Removed DimensionValues query (no need to have a separate query to fetch dimension values, as we can just specify one dimensionIri in DataCubesComponents to get the same result)
• Removed dimensions & measures from SearchCubes query
• We will now get all hierarchy levels, not limited to 6 as before (by using scalar instead of nested retrieval)
• To further improve performance and limit resolver chains, I've extracted the dimension types from GQL to TypeScript. It also means that we now explicitly parse them inside the resolver
• Removed DimensionMetadataFragment type, which means we can now better type Dimension and Measure, instead of mostly using the type that contains both

Misc

• Fixed default empty filter selection when initializing a segment and hierarchy is not present

---

Context

This PR is a first part of preparations for merging the cubes. For the technical concept, see this Notion document.

• DataCubesComponents
• DataCubesMetadata
• DataCubesObservations

How to test

1. Click around in the application and make sure nothing is broken 💥
2. Open GQL playground and send e.g. the following query to see that all dimensions from both cubes were fetched.

Query

query DataCubesComponents(
  $sourceType: String!
  $sourceUrl: String!
  $locale: String!
  $filters: [DataCubeFilter!]!
) {
  dataCubesComponents(
    sourceType: $sourceType
    sourceUrl: $sourceUrl
    locale: $locale
    filters: $filters
  )
}

Variables

{
  "sourceType": "sparql",
  "sourceUrl": "https://lindas.admin.ch/query",
  "locale": "en",
  "filters": [{ "iri": "https://energy.ld.admin.ch/sfoe/bfe_ogd84_einmalverguetung_fuer_photovoltaikanlagen/9", "latest": true }, { "iri": "https://environment.ld.admin.ch/foen/ubd0104/6", "latest": true }]
}

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
visualization-tool	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 15, 2023 9:18am

[Rdataflow]

@bprusinowski can you add an additional point for how to test CI screenshots 😄

[bprusinowski]

Hi @Rdataflow, in fact we would need to switch back to Argos for screenshot testing, as we used the limit already on Percy (so not possible to check them on CI now) 😞

[ptbrowne]

I followed the direction and clicked around : I am getting a 405 on "Verkehrsleistung auf den Flugplätzen", when going to the table. Not sure this is related to the PR.

[bprusinowski]

@ptbrowne it looks like the payload is too large (https://vercel.com/ixt/visualization-tool/3pbpsBYim6U2gRc2bE1eDnDovxMx/logs?page=2&timeline=past30Minutes&startDate=1700148221941&endDate=1700150021941&selectedLogId=1700149691375354119628444121&selectedLogTimestamp=1700149691375&forceEndDate=1700149937685), the underlying message says it's error 413 👀

[ptbrowne]

I will retake a look tomorrow.

What I like

• Simplifying resolver chain
• Less different type of queries
• Better typing client side

What I have questions on

• Why we put everything into 1 query client side ? Seems like it would not be as cache friendly & we would have unwanted invalidations when adding a new cube to compare.
  • To solve this problem, seems like the client hooks could hide that they are doing multiple graphql requests & merge everything client side. We use this kind of technique on UIS.
  • Technically, useDataCubesComponentsQuery could be a custom hook relying on
    multiple useDataCubesComponentQuery urql hook calls (without "s") calls

[ptbrowne]

Nit: This could be done in the dimensionValuesLoader right ?

[bprusinowski]

Yup, this could be moved there 👍

[ptbrowne]

Was wondering if cubeFilters could not be more explicit as we already use filters for value filters ?

[bprusinowski]

Yes, this name should be more clear :)

[bprusinowski]

Thanks for reviewing @ptbrowne. I agree it would be better to separate the components queries per cube, so cube-specific queries are not re-fetched when adding another one 👍 I would be interested to learn how to do it, I guess we would need to use client.query directly instead of current useXYZQuery hook so we have several queries combined together in Promise.all?