Widget Dashboard Queries using extracted Search Parameters #179
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Widget UI/UX is pretty broken though.
…allly (from JSON spec) change all widgets to depend on dashboard-widget.
ensure we can customize the columns for the table widget, and charts.
make sure all widgets suppport CommonModule. Remove useless manual legend from Donut chart.
Dyanmically generating data for dual guages widget.
- added fields tracking SearchParameters During record insertion, populating FHIR Resource tables, & populating search param fields using FhirPathjs + GoJa Library Moved the QueryResources function into its own file. Added an automigrate function for the new db models working on query parsing logic (with raw SQL).
…ed files will change every time) make sure simple field types are not json stringified when returned from GoJa.
BROKEN - token, quantities, references, etc.
…ize) Working Quantity Json parsing - added support for FROM clauses (used by json_each)
Limited to only a single use (cant be paird with other clauses). added explicit Security annotations for required SQL injection protection.
…FHIRPath expressions. fixing this so it works for complex and primitive types.
…n parameters at the same time. Removing restriction.
…ter, ie. (value = "foo" OR value = "bar") AND (value = "baz")
renamed RawResource field to ResourceRaw to be consistent. Using `resource_raw` json key to be consistent with UI.
dashboard widget where clause changed to support FHIR Search API style querying (server side instead of client side)
… FHIR Table name by resource type. ListResources now uses the FHIR Resource tables.
Make sure we continue to use models.ResourceFhir{} when querying.
Adding Select and Group statements to QueryResources GORM query:
- Select fhir.* is required, otherwise json_each "tables" are included in the result, and mapping to ResourceFhir struct fails (json_each.id is a number not a UUID)
- Group By fhir.id is required otherwise we can have multiple entries for the same resource (json_each can match multiple times).
…oss every table).
fixing condition display in medical history page.
Update all SqliteRepository funtions to correctly search against the individual tables, but use ResourceBase as the query result
…th individual table setup)
regenerate mocked sqlite database struct Adding tests for composition
adding more testing for CustomUser Fixing broken tests removed unnecessary API calls from dashboard component. TODO: storybook tests are still broken.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
parsingkey to display data in charts.Just wanted to give you all an update to this earlier post. There's been alot of progress on the Widget Based Dashboard implementation:
The database schema has changed significantly: every FHIR resource type is now stored in its own table. This allows us to create additional columns for resource specific search parameter values.
the new DB schema is incompatible with the old one, so you will need to delete your old .db file if you've presisted it outside of the container. Fasten will eventually have a DB migration tool, but that has not been implemented yet
Fasten can now search against most of the Search Paramter types ( token, string, quantity, number, date, uri)
All the DB Queries have been correctly updated to handle this new schema (querying accross multiple tables)
Search Parameters queries support AND, OR logic ( ie. bloodpressure > 100 AND bloodpressure < 150, etc)
Almost complete test coverage has been added to the database layer.
Documentation for the Dashboard Configuration & Query language has been started (still a WIP) - https://docs.fastenhealth.com/dashboard/
Default Dashboard can now be overridden with a custom JSON configuration file. (instructions for how to do this will be added soon) - https://github.com/fastenhealth/fasten-onprem/blob/dynamic_db_models/backend/pkg/web/handler/dashboard/default.json
You can see the full list of changes in this PR (and test them out yourselves) - #177
closes #156
closes #21
As discussed previously, there are multiple issues with the current query syntax used by the Widgets. In short, the query is processed in two places, with a majority of the processing taking place in the user's browser -- which means a large amount of data is retrieved and transmitted from the database.
After a very productive conversation with @cody from Medplum, I've decided to mimic their solution - extract the FHIR SearchParameters for each resource as they are being stored, and store them beside the raw FHIR resource data. The complexity here is that there does not seem to be a working Go implementation of the FHIRPath.js library, so instead we'll be using github.com/dop251/goja to run Javascript code in Go 😬 .
I can confirm that it works, with a reasonable amount of overhead (this Javascript processing only takes place during record storage, so it's a 1 time expense). The benefits are immediate: querying is now much more performant, with a "well documented" syntax (we'll be using a subset of the FHIR search syntax) -- hopefully fully unlocking the widget-based dashboard's potential, allowing developers to create & share their condition specific dashboards with non-developers.
related #21
related #156
Search Safety Checklis - https://hl7.org/fhir/R4/safety.html#search
TODO:
stringsearch needs to support Human Names correctly