Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(robot-server): Refactor hardware wrapper and other lifetime dependencies #8213

Closed
wants to merge 49 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
e77dd78
Create `slow_initializing` module.
SyntaxColoring Jul 29, 2021
686b1fa
Make `API.register_callback()` non-async.
SyntaxColoring Aug 3, 2021
3e89b33
Eliminate HardwareWrapper.
SyntaxColoring Aug 4, 2021
fca92e0
WIP: Comment out tests.
SyntaxColoring Aug 9, 2021
752fabb
WIP.
SyntaxColoring Aug 9, 2021
656a100
Work around `request` being inaccessible from WebSocket endpoints.
SyntaxColoring Aug 9, 2021
0f2ea16
Rename hardware_wrapper.py -> hardware_initialization.py.
SyntaxColoring Aug 9, 2021
d7b29eb
Make _make_event_publisher() non-async, and leave .close() todo.
SyntaxColoring Aug 9, 2021
bf04ca4
Cleanup.
SyntaxColoring Aug 9, 2021
4dfa729
Uncomment and start fixing up tests.
SyntaxColoring Aug 9, 2021
bf31cc0
Todo for hardware shutdown.
SyntaxColoring Aug 9, 2021
e6bc2e4
Make _init_event_watchers non-async.
SyntaxColoring Aug 9, 2021
ab183c3
Port test_slow_initializing.py to Decoy.
SyntaxColoring Aug 10, 2021
3e0f901
Delete unnecessary comments about BaseException.
SyntaxColoring Aug 10, 2021
e3eb9bd
WIP: refactor app lifetime dependencies.
SyntaxColoring Aug 4, 2021
9428465
Tests: override motion lock dependency with a mock.
SyntaxColoring Aug 4, 2021
6a33398
Documentation and minor cleanup.
SyntaxColoring Aug 4, 2021
c389c36
Rename app_dependencies.py to lifetime_dependencies.py.
SyntaxColoring Aug 10, 2021
7c62bf8
Improve module docstring.
SyntaxColoring Aug 10, 2021
bcb6397
More WIP of lifetime dependencies.
SyntaxColoring Aug 10, 2021
f2d54fb
Clarify comment.
SyntaxColoring Aug 11, 2021
e42916b
Move some stuff around.
SyntaxColoring Aug 11, 2021
602eb75
Cleanup, and fix circular import.
SyntaxColoring Aug 11, 2021
0afd56a
Formatting.
SyntaxColoring Aug 11, 2021
c0dfca8
Linting.
SyntaxColoring Aug 11, 2021
c818120
Delete unnecessary comment.
SyntaxColoring Aug 11, 2021
697e2fd
Format and lint.
SyntaxColoring Aug 11, 2021
5ab454b
Rewrite `HardwareWrapper` tests for `hardware_initialization` module.
SyntaxColoring Aug 12, 2021
1aa55bd
Format and lint.
SyntaxColoring Aug 11, 2021
6552b3c
Merge lifetime dependency refactors.
SyntaxColoring Aug 12, 2021
774cfb4
Lint.
SyntaxColoring Aug 12, 2021
3a95844
Override with a *real* motion lock, to preserve existing behavior.
SyntaxColoring Aug 13, 2021
b6d95af
Port ProtocolManager and SessionManager cleanup code.
SyntaxColoring Aug 13, 2021
3b854b2
Delete obsolete comment.
SyntaxColoring Aug 13, 2021
dc5032d
Handle `InitializationOngoingError` in `get_rpc_server()` (sort of).
SyntaxColoring Aug 13, 2021
b7ac395
Clean up some of `lifetime_dependencies`' imports.
SyntaxColoring Aug 13, 2021
8827b2f
Delete unnecessary todo comment.
SyntaxColoring Aug 13, 2021
3573b80
Don't unnecessarily wrap `motion_lock` in a context manager.
SyntaxColoring Aug 13, 2021
59d46f3
Clarify a comment.
SyntaxColoring Aug 13, 2021
615e06a
Remove now-unnecessary `@call_once` decorators.
SyntaxColoring Aug 13, 2021
8d06237
Remove print debug statements.
SyntaxColoring Aug 14, 2021
abd5d19
Remove unused import.
SyntaxColoring Aug 14, 2021
72a47dd
Remove resolved todo.
SyntaxColoring Aug 16, 2021
1c8d3b9
Rename `lifetime_dependency_set` args to `lifetime_dependencies`.
SyntaxColoring Aug 17, 2021
84a6ff6
Add some docstrings to `lifetime_dependencies` internal functions.
SyntaxColoring Aug 17, 2021
a960dd1
Make small improvements to how we store dependencies on `app`.
SyntaxColoring Aug 18, 2021
b3593da
Fix startup accidentally waiting for `ThreadManager` to be ready.
SyntaxColoring Aug 18, 2021
233ef3d
Reformat.
SyntaxColoring Aug 18, 2021
5343b36
Make `API.register_callback()` async again.
SyntaxColoring Aug 18, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion robot-server/.flake8
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,4 @@ per-file-ignores =
tests/service/*:ANN,D
tests/conftest.py:ANN,D
tests/test_app.py:ANN,D
tests/test_hardware_wrapper.py:ANN,D
tests/test_util.py:ANN,D
2 changes: 1 addition & 1 deletion robot-server/robot_server/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
This server provides the main control interface for an Opentrons robot.
"""

from .app import app
from .app_setup import app
Copy link
Contributor Author

@SyntaxColoring SyntaxColoring Aug 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ This rename avoids ambiguity in from robot_server import app. (Would it import the app FastAPI object, or the app module?)


__all__ = [
"app",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""Main FastAPI application."""
import logging


from opentrons import __version__
from fastapi import FastAPI
Expand All @@ -8,21 +8,15 @@
from starlette.requests import Request
from starlette.middleware.base import RequestResponseEndpoint

from .service.dependencies import (
get_rpc_server,
get_protocol_manager,
get_hardware_wrapper,
get_session_manager,
)

from .errors import exception_handlers
from .router import router
from .service import initialize_logging
from . import lifetime_dependencies
from . import constants

log = logging.getLogger(__name__)


# Our global ASGI app object. Our ASGI server (currently Uvicorn) finds this and
# sends web requests to it.
app = FastAPI(
title="Opentrons OT-2 HTTP API Spec",
description=(
Expand All @@ -35,13 +29,15 @@
version=__version__,
)


# exception handlers
# TODO(mc, 2021-05-10): after upgrade to FastAPI > 0.61.2, we can pass these
# to FastAPI's `exception_handlers` arg instead. Current version has bug, see:
# https://github.com/tiangolo/fastapi/pull/1924
for exc_cls, handler in exception_handlers.items():
app.add_exception_handler(exc_cls, handler)


# cors
app.add_middleware(
CORSMiddleware,
Expand All @@ -51,31 +47,9 @@
allow_headers=["*"],
)

# main router
app.include_router(router=router)


@app.on_event("startup")
async def on_startup() -> None:
"""Handle app startup."""
initialize_logging()
# Initialize api
(await get_hardware_wrapper()).async_initialize()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignoring the fact that it was broken, I dislike this removal. If I'm trying to establish context while reading this server code, the question "how is the hardware connection created?" is really important, and when we put it here, in the app-level startup handler, the pathway to find that answer is really obvious



@app.on_event("shutdown")
async def on_shutdown() -> None:
"""Handle app shutdown."""
s = await get_rpc_server()
await s.on_shutdown()
# Remove all sessions
await (await get_session_manager()).remove_all()
# Remove all uploaded protocols
(await get_protocol_manager()).remove_all()


@app.middleware("http")
async def api_version_response_header(
async def _api_version_response_header(
request: Request,
call_next: RequestResponseEndpoint,
) -> Response:
Expand All @@ -90,3 +64,14 @@ async def api_version_response_header(
response.headers[constants.API_VERSION_HEADER] = str(request.state.api_version)
response.headers[constants.MIN_API_VERSION_HEADER] = str(constants.MIN_API_VERSION)
return response


# main router
app.include_router(router=router)


app.on_event("startup")(initialize_logging)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment from @mcous:

This method is "discouraged" according to the startlette docs:

# The following usages are now discouraged in favour of configuration
#  during Starlette.__init__(...)
def on_event(self, event_type: str) -> typing.Callable:
    return self.router.on_event(event_type)

Copy link
Contributor Author

@SyntaxColoring SyntaxColoring Aug 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is FastAPI's @on_event("startup"), not Starlette's.

My understanding based on reading a bunch of GitHub threads is:

  • FastAPI's on_event("startup") has historically been basically a straight pass-through to Starlette's on_event("startup").
  • Starlette has been moving away from that API, and towards:
    • Accepting startup/shutdown handlers up-front, as arguments to __init__().
    • Accepting a single lifetime context manager instead of split startup and shutdown functions.
  • FastAPI, meanwhile, has not caught up to those Starlette changes. I think they're still trying to work out how FastAPI's automatic dependency resolution machinery would fit in. So, for now, this is what FastAPI gives us.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is FastAPI's @on_event("startup"), not Starlette's.

From reading the source, FastAPI does not define on_event; it's definitely a straight pass-through because the FastAPI class inherits from Starlette.

I think using the __init__ methods (which FastAPI also accepts) is the best available move. If we must use @app.on_event (which may turn out to be the case depending on how badly we want the app.state dictionary in there), I think all of them should be defined in the top-level app.py

lifetime_dependencies.install_startup_shutdown_handlers(app)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing app as an argument to a function that's going to mutate app is a warning sign to me. From what I can tell, lifetime_dependencies requires two things in the current PR:

  1. app, to mutate the startup and shutdown handlers
  2. app.state, to attach persistent data

I think (1) should be removed in favor of the app setup being in charge of attached handlers exposed by the lifetime_dependencies module, and any other methods should operate on a State object rather than a FastAPI app



__all__ = ["app"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These __all__ variables in internal, non-init files feel a little redundant to me

83 changes: 83 additions & 0 deletions robot-server/robot_server/hardware_initialization.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
"""Utilities for initializing the hardware interface."""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Utilities" is always a bit of a warning sign for me. Something you need utilities, but if we have a hardware connection we need to initialize and hand out as a dependency, shouldn't those all be in the same place?

Is there a way we can restructure this PR to have robot_server/hardware.py be the thing that actually exports a startup / shutdown handler for the hardware along with the get_hardware dependency function? I'm imagining a system where any given dependency is able to use classes / functions in lifetime_dependencies to declare themselves a lifetime dependency.

For example, if this module was to export 2 or 3 methods, I think everything would be a lot easier to follow:

  1. def initialize_hardware(app_state: State) -> None
    • Creates a backround task to initialize a ThreadManager
    • Uses slow_initializing and/or lifetime_dependencies to attach it to app state
    • (Maybe) registers cleanup with lifetime_dependencies
  2. def get_hardware(app_state: State) -> ThreadManager
    • Retrieve the slow initializing hardware
    • Raise if not initialized
  3. (Maybe) def shutdown_hardware(app_state: State) -> None
    • Shut down the hardware
    • The necessity of this would depend on if lifetime_dependencies centralized cleanup or not



import logging
from pathlib import Path

from opentrons import ThreadManager, initialize as initialize_api
from opentrons.hardware_control.simulator_setup import load_simulator
from opentrons.hardware_control.types import HardwareEvent, HardwareEventType
from opentrons.util.helpers import utc_now

from notify_server.clients.publisher import Publisher
from notify_server.models import event, topics
from notify_server.models.hardware_event import DoorStatePayload

from .settings import get_settings


log = logging.getLogger(__name__)


async def initialize(event_publisher: Publisher) -> ThreadManager:
"""Initialize the API."""
# todo(mm, 2021-08-12): It would be easier for tests to check that this function
# can successfully initialize simulating hardware if app_settings were passed in
# as an argument, instead of being fetched here.
app_settings = get_settings()
thread_manager: ThreadManager
if app_settings.simulator_configuration_file_path:
log.info(
f"Loading simulator from "
f"{app_settings.simulator_configuration_file_path}"
)
# A path to a simulation configuration is defined. Let's use it.
thread_manager = ThreadManager(
load_simulator, Path(app_settings.simulator_configuration_file_path)
)
else:
# Create the hardware
thread_manager = await initialize_api()

log.info("Starting hardware-event-notify publisher")
door_event_forwarder = DoorEventForwarder(event_publisher)
# fixme(mm, 2021-08-12): This might be a typing error. forward() can only
# take a HardwareEvent, but it looks like ThreadManager can also pass
# other things as the argument to a callback?
await thread_manager.register_callback(door_event_forwarder.forward)

log.info("Opentrons API initialized")
return thread_manager


class DoorEventForwarder:
"""Forwards front door open/close events to a notify-server publisher."""

def __init__(self, publisher: Publisher) -> None:
"""Initialize the `DoorEventForwarder`.

Params:

publisher: Where subsequent calls to `forward_hardware_event` will forward
to.
"""
self.publisher = publisher

def forward(self, hardware_event: HardwareEvent) -> None:
"""Forward `hardware_event` if it's a door event.

Otherwise, no-op.
"""
if hardware_event.event == HardwareEventType.DOOR_SWITCH_CHANGE:
payload = DoorStatePayload(state=hardware_event.new_state)
else:
return
topic = topics.RobotEventTopics.HARDWARE_EVENTS
publisher = "robot_server_event_publisher"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ The publisher value changed because it was previously based on the name of the containing class, which no longer exists.

event_to_publish = event.Event(
createdOn=utc_now(), publisher=publisher, data=payload
)
self.publisher.send_nowait(topic, event_to_publish)


__all__ = ["initialize", "DoorEventForwarder"]
86 changes: 0 additions & 86 deletions robot-server/robot_server/hardware_wrapper.py

This file was deleted.

Loading