Export API for a system level export returning one resource per resource type #2005
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ |
mattwiller
reviewed
May 9, 2023
tonylee80
added
the
fhir-datastore
tonylee80
commented
May 9, 2023
| while (hasMore) { | ||
| const bundle = await repo.search({ | ||
| resourceType, | ||
| count: 1000, | ||
| offset, |
There was a problem hiding this comment.
addresses infinite loop when there are more than 1000 resources for a resource type
tonylee80
commented
May 9, 2023
| for (const entry of bundle.entry) { | ||
| if (entry.resource?.id) { | ||
| writes.push(exporter.writeResource(entry.resource)); | ||
| await exporter.writeResource(entry.resource); |
There was a problem hiding this comment.
removing using 'Promise.all'. Each time export.writeResource was called this.writers was always {}. A new writer for each resourceType was always being created at
tonylee80
commented
May 9, 2023
| @@ -34,7 +34,7 @@ export async function bulkExportHandler(req: Request, res: Response): Promise<vo | |||
| if (!canBeExported(resourceType) || (types && !types.includes(resourceType))) { | |||
| continue; | |||
| } | |||
| await exportResourceType(exporter, project, resourceType as ResourceType); | |||
There was a problem hiding this comment.
project was unused
tonylee80
commented
May 9, 2023
| @@ -44,30 +44,40 @@ export async function bulkExportHandler(req: Request, res: Response): Promise<vo | |||
| res.set('Content-Location', `${baseUrl}fhir/R4/bulkdata/export/${bulkDataExport.id}`).status(202).json(accepted); | |||
| } | |||
|
|
|||
| async function exportResourceType(exporter: BulkExporter, project: Project, resourceType: ResourceType): Promise<void> { | |||
There was a problem hiding this comment.
project was unused
mattwiller
approved these changes
May 9, 2023
tonylee80
commented
May 10, 2023
| @@ -44,30 +44,40 @@ export async function bulkExportHandler(req: Request, res: Response): Promise<vo | |||
| res.set('Content-Location', `${baseUrl}fhir/R4/bulkdata/export/${bulkDataExport.id}`).status(202).json(accepted); | |||
| } | |||
|
|
|||
| async function exportResourceType(exporter: BulkExporter, project: Project, resourceType: ResourceType): Promise<void> { | |||
| export async function exportResourceType( | |||
There was a problem hiding this comment.
adding export and param maxResources to unit test pagination.
tonylee80
commented
May 10, 2023
mattwiller
previously approved these changes
May 10, 2023
codyebberson
requested changes
May 10, 2023
This reverts commit 5c37862.
|
Kudos, SonarCloud Quality Gate passed! |
codyebberson
approved these changes
May 11, 2023
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fixes export returning one resource per resource type and fixes infinite loop when there are greater than 1000 resources for a resource type
🤖 Generated by Copilot at 5c37862
Summary
🆕🐛📄
Enhanced the bulk export operation to support pagination and resource limits, and added tests to verify the functionality. Fixed a bug in
exportResourceTypethat affected the output files.Walkthrough
exportResourceTypefunction to accept amaxResourcesparameter and anoffsetvariable, and to iterate through paginated search results by following thenextlink in the search bundle (link, link)exportResourceTypefunction that caused some resources to be skipped or written to the wrong file by calling thegetWriterfunction on the exporter before writing each resource (link)projectparameter from theexportResourceTypefunction call in thebulkExportHandlerfunction (link)export.test.tsfile to test the functionality of exporting multiple resources by resource type and iterating through paginated search results, using therequest,waitFor, andexpectlibraries, and theinitTestAuth,createTestProject, andcreateReferencefunctions (link)ObservationandPatienttypes from@medplum/fhirtypes, and thesystemRepo,exportResourceType,BulkExporter, andcreateReferencefunctions from their respective modules, to be used in the new test cases (link, link)export.tsfile to match the order of the imports in theexport.test.tsfile, for consistency and readability (link)