Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix-3238 gracefully shutdown server #3239

Merged

Conversation

dillonstreator
Copy link
Contributor

Fixes #3238

@dillonstreator dillonstreator requested a review from a team as a code owner November 6, 2023 15:49
Copy link

vercel bot commented Nov 6, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

2 Ignored Deployments
Name Status Preview Comments Updated (UTC)
medplum-storybook ⬜️ Ignored (Inspect) Visit Preview Nov 8, 2023 7:22pm
medplum-www ⬜️ Ignored (Inspect) Visit Preview Nov 8, 2023 7:22pm

Copy link

vercel bot commented Nov 6, 2023

@dillonstreator is attempting to deploy a commit to the Medplum Team on Vercel.

A member of the Team first needs to authorize it.

Copy link
Contributor

sweep-ai bot commented Nov 6, 2023

Apply Sweep Rules to your PR?

  • Apply: All new business logic should have corresponding unit tests.
  • Apply: Refactor large functions to be more modular.

@reshmakh reshmakh modified the milestone: November 30, 2023 Nov 6, 2023
@coveralls
Copy link

coveralls commented Nov 6, 2023

Coverage Status

coverage: 93.92% (-0.009%) from 93.929%
when pulling 4538d06 on dillonstreator:fix-3238-server-graceful-shutdown
into f3c7d5f on medplum:main.

@codyebberson
Copy link
Member

@dillonstreator - Sorry about the build issues... It looks like it's something funky with Coveralls. I have a fix here: #3246

@@ -264,6 +265,7 @@ function addDefaults(config: MedplumServerConfig): MedplumServerConfig {
config.botLambdaLayerName = config.botLambdaLayerName || 'medplum-bot-layer';
config.bcryptHashSalt = config.bcryptHashSalt || 10;
config.bullmq = { removeOnComplete: { count: 1 }, removeOnFail: { count: 1 }, ...config.bullmq };
config.shutdownTimeoutMilliseconds = config.shutdownTimeoutMilliseconds ?? 30000;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is redundant in its current state as the default timeout for http-graceful-shutdown is already 30 seconds https://github.com/sebhildebrandt/http-graceful-shutdown#options

I can remove, unless we'd like to adjust the default.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

30 seconds is appropriate. I don't have strong feelings about whether to be explicit or implicit here.

@codyebberson
Copy link
Member

For my own curiosity, the AWS Fargate Task timeout is also 30 seconds: https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_StopTask.html

In the common case, server shutdown should be fast (under 1 second). The exceptions being:

  1. Long running HTTP requests, such as a large FHIR transaction bundle
  2. Long running BullMQ jobs, such as "rebuild structure definitions", "reindex resources", etc.

Those can take upwards of 30 seconds.

Overall, it feels good to be explicit. Thanks @dillonstreator

@@ -22,6 +23,17 @@ export async function main(configName: string): Promise<void> {
const server = app.listen(config.port);
server.keepAliveTimeout = config.keepAliveTimeout ?? 90000;
globalLogger.info('Server started', { port: config.port });
gracefulShutdown(server, {
timeout: config.shutdownTimeoutMilliseconds,
development: process.env.NODE_ENV !== "production",
Copy link
Contributor Author

@dillonstreator dillonstreator Nov 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@codyebberson might be nice for the app config to define environment with an enum.
Maybe a separate issue?

@codyebberson
Copy link
Member

@dillonstreator - Apologies for the inconvenience, but if you rebase & resubmit, the Github Actions problems should be fixed. Or let me know, and we can do it on your behalf.

@dillonstreator dillonstreator force-pushed the fix-3238-server-graceful-shutdown branch from 4538d06 to 833067c Compare November 7, 2023 00:45
@dillonstreator
Copy link
Contributor Author

@dillonstreator - Apologies for the inconvenience, but if you rebase & resubmit, the Github Actions problems should be fixed. Or let me know, and we can do it on your behalf.

All good - I've rebased on upstream/main and we should now be set 👍

@codyebberson
Copy link
Member

Blerg, the prettier comment tool doesn't work on forks apparently, sorry about that. There's one prettier error in index.ts. We'd be happy to fix it as well:

https://github.com/medplum/medplum/actions/runs/6778421837/job/18429916344?pr=3239

image

@codyebberson codyebberson added this pull request to the merge queue Nov 9, 2023
Merged via the queue into medplum:main with commit 02328da Nov 9, 2023
12 checks passed
@reshmakh reshmakh added this to the November 30, 2023 milestone Nov 9, 2023
codyebberson added a commit that referenced this pull request Nov 9, 2023
chore(actions): add console output for `prettier-fmt` Actions workflow (#3278)
feat(auth/external): make `PKCE` on external auth optional (#3279)
server/fhir/repo execute independent io concurrently (#3227)
MeasureReportDisplay   (#3242)
Create documentation on creating test data using batches (#3247)
Clarify project admin vs. Super admin (#3267)
feat(fhircast): add `FHIRcast` OAuth 2.0 scopes (#3274)
Adding Health Gorilla setup documentation and samples (#3250)
fix-3238 gracefully shutdown server (#3239)
github-merge-queue bot pushed a commit that referenced this pull request Nov 9, 2023
* Release Version 2.1.12

chore(actions): add console output for `prettier-fmt` Actions workflow (#3278)
feat(auth/external): make `PKCE` on external auth optional (#3279)
server/fhir/repo execute independent io concurrently (#3227)
MeasureReportDisplay   (#3242)
Create documentation on creating test data using batches (#3247)
Clarify project admin vs. Super admin (#3267)
feat(fhircast): add `FHIRcast` OAuth 2.0 scopes (#3274)
Adding Health Gorilla setup documentation and samples (#3250)
fix-3238 gracefully shutdown server (#3239)

* Fix package-lock.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Gracefully shut down main packages/server process
4 participants