feat(system-server): add sqlite database and barebones HTTP server

[fsinapi]

Overview

This PR adds a SQLite database and an endpoint-less HTTP server to the system-server project.

• The system server uses a persistent directory to store the database (and in a future PR, the UUID for signing JWT's). The directory defaults to /var/lib/opentrons-system-server but can be overwritten with an environment variable like on the robot-server.
• The database contains one table, registration, which is intended to store long-term app registrations.
  • The database is configured as specified in RCORE-508 with the addition of a UNIQUE clause for the tuple of (subject, agent, agent-id) in order to simplify the logic for interacting with the database.
• The database contains a second table, migration, for tracking any future database migrations needed by the system-server.
• An HTTP server is configured on port 32950 by default.
  • At least based on looking through our repos and probing that port from an SSH session on a robot, this is unused until now.

Test Plan

This can be tested on an OT-2 with make-push since all the new python packages are already on the disk from the robot-server. To test on an OT-3, a new image has to be flashed because many of these libraries are only in /opt/opentrons-robot-server.

After getting this on a robot, ssh in and do the following:

• Check with `systemctl status opentrons-system-server`` that the service is running

• Check that /var/lib/opentrons-system-server has been created and system_server.db exists

• curl a PUT request to localhost:32950/system/refresh and make sure you get the text 'OK' and the persistent DB is generated.

• Talk to the robot through an app just to make sure the robot-server and update-server are still working fine

• Test on OT-2

• Test on OT-3

Changelog

• Added persistence submodule to create a persistent directory and generate a SQL database
• Added settings submodule to get environment variable configuration settings
• Added app_setup and app_state for barebones FastAPI HTTP server configuration, with router.py as a placeholder router that does nothing so far.
• Added a TEMPORARY endpoint that initializes the persistent directory & database. This is not necessarily safe but it provides an avenue to verify that db generation works correctly.
• Added an environment variable for OT_SYSTEM_SERVER_persistent_directory to the OT-2 systemd unit file. This has to be added separately in oe-core for OT-3.

Review requests

• Would it make sense to just get rid of the environment variable settings? I mostly pulled it in because that seems to be how we prefer configuring the robot-server, but since this server already has CLI options I think I might prefer to just put the persistent directory into the CLI options.
  • I came to the conclusion that this is the more elegant way to handle settings that are needed during FastAPI runtime.

Risk assessment

Low, as long as this doesn't interfere with other servers then it has no user-facing effects yet.

[codecov]

Codecov Report

Merging #12085 (ef791b4) into edge (ed0e6f3) will decrease coverage by 0.51%.
The diff coverage is 76.43%.

@@            Coverage Diff             @@
##             edge   #12085      +/-   ##
==========================================
- Coverage   74.48%   73.97%   -0.51%     
==========================================
  Files        2180     2211      +31     
  Lines       59867    61071    +1204     
  Branches     6467     6477      +10     
==========================================
+ Hits        44589    45176     +587     
- Misses      13747    14358     +611     
- Partials     1531     1537       +6     

Flag	Coverage Δ
system-server	67.90% <76.43%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
system-server/system_server/__main__.py	0.00% <0.00%> (ø)
system-server/system_server/cli.py	0.00% <0.00%> (ø)
system-server/system_server/_version.py	61.53% <61.53%> (ø)
system-server/system_server/app_setup.py	71.42% <71.42%> (ø)
...ystem-server/system_server/persistence/__init__.py	73.33% <73.33%> (ø)
system-server/system_server/settings/settings.py	77.27% <77.27%> (ø)
system-server/system_server/router.py	85.71% <85.71%> (ø)
...ystem-server/system_server/persistence/database.py	86.36% <86.36%> (ø)
system-server/system_server/app_state.py	86.66% <86.66%> (ø)
system-server/system_server/__init__.py	100.00% <100.00%> (ø)
... and 63 more

[sfoster1]

Looks solid to me!

[SyntaxColoring]

Exciting! Various bits of feedback learned from robot-server lessons.

[SyntaxColoring]

I don't think we should call get_persistence_directory() and get_sql_engine() like this.

In robot-server, where these function's definitions are copied from, these aren't normal functions—they're FastAPI dependency functions.

We need to draw a bright line between the two kinds of function and treat them as mutually exclusive. When you call a FastAPI dependency function as if it were a normal function, it's very easy to accidentally subvert static type safety and cause confusing breakage. This is a FastAPI trap that we (and others) have been bitten by before—see #8051.

In principle, I think eagerly initializing the database is the right thing to do, and we're moving in this direction in robot-server (#11615). But our code for this in robot-server is a highly-duplicated mess right now, so I don't think we have a good model for eager initialization just yet.

For the short term, what do you think about doing the default FastAPI thing and let the database be initialized lazily for the first HTTP request that requires it? We got away with that in robot-server for a while.

[fsinapi]

Yeah that makes sense even if it makes me Big Sad.

I'm gonna add a fake little endpoint that depends on the database so it's possible to test the db generation.

[mcous]

I agree with all of Max's feedback. I also have some specific feedback on the tests added in test_persistent_generation

[mcous]

IMO this test inappropriately mixes mocking, implementation details, and logic testing. It'll likely be brittle over time and doesn't really test that things work, due to the mocks. It also brings in the entire world of FastAPI.

I recommend restructuring this so that the question "is it going to the expected directory?" is tested without mocks. For example, the suite could have:

• Test that checks the settings module has the correct directory value
• Test that checks the persistence module, given a value from settings, creates a directory at the given value
• Test, with mocks, that checks the settings module and persistence module are wired up to each other properly, but does not test any logic because mocks inherently compromise logic tests
• Acceptance test that does not use mocks and simply tests that resources are persisted.
  • Thanks to the other tests, we don't really need to check in this test if they're persisted in the correct place, because that's covered by more specific tests

[SyntaxColoring]

One file leakage thing. Other than this, changes since my last review look good. Thanks!