Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(robot-server): Refactor hardware wrapper and other lifetime dependencies #8213

Closed
wants to merge 49 commits into from
Closed
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
e77dd78
Create `slow_initializing` module.
SyntaxColoring Jul 29, 2021
686b1fa
Make `API.register_callback()` non-async.
SyntaxColoring Aug 3, 2021
3e89b33
Eliminate HardwareWrapper.
SyntaxColoring Aug 4, 2021
fca92e0
WIP: Comment out tests.
SyntaxColoring Aug 9, 2021
752fabb
WIP.
SyntaxColoring Aug 9, 2021
656a100
Work around `request` being inaccessible from WebSocket endpoints.
SyntaxColoring Aug 9, 2021
0f2ea16
Rename hardware_wrapper.py -> hardware_initialization.py.
SyntaxColoring Aug 9, 2021
d7b29eb
Make _make_event_publisher() non-async, and leave .close() todo.
SyntaxColoring Aug 9, 2021
bf04ca4
Cleanup.
SyntaxColoring Aug 9, 2021
4dfa729
Uncomment and start fixing up tests.
SyntaxColoring Aug 9, 2021
bf31cc0
Todo for hardware shutdown.
SyntaxColoring Aug 9, 2021
e6bc2e4
Make _init_event_watchers non-async.
SyntaxColoring Aug 9, 2021
ab183c3
Port test_slow_initializing.py to Decoy.
SyntaxColoring Aug 10, 2021
3e0f901
Delete unnecessary comments about BaseException.
SyntaxColoring Aug 10, 2021
e3eb9bd
WIP: refactor app lifetime dependencies.
SyntaxColoring Aug 4, 2021
9428465
Tests: override motion lock dependency with a mock.
SyntaxColoring Aug 4, 2021
6a33398
Documentation and minor cleanup.
SyntaxColoring Aug 4, 2021
c389c36
Rename app_dependencies.py to lifetime_dependencies.py.
SyntaxColoring Aug 10, 2021
7c62bf8
Improve module docstring.
SyntaxColoring Aug 10, 2021
bcb6397
More WIP of lifetime dependencies.
SyntaxColoring Aug 10, 2021
f2d54fb
Clarify comment.
SyntaxColoring Aug 11, 2021
e42916b
Move some stuff around.
SyntaxColoring Aug 11, 2021
602eb75
Cleanup, and fix circular import.
SyntaxColoring Aug 11, 2021
0afd56a
Formatting.
SyntaxColoring Aug 11, 2021
c0dfca8
Linting.
SyntaxColoring Aug 11, 2021
c818120
Delete unnecessary comment.
SyntaxColoring Aug 11, 2021
697e2fd
Format and lint.
SyntaxColoring Aug 11, 2021
5ab454b
Rewrite `HardwareWrapper` tests for `hardware_initialization` module.
SyntaxColoring Aug 12, 2021
1aa55bd
Format and lint.
SyntaxColoring Aug 11, 2021
6552b3c
Merge lifetime dependency refactors.
SyntaxColoring Aug 12, 2021
774cfb4
Lint.
SyntaxColoring Aug 12, 2021
3a95844
Override with a *real* motion lock, to preserve existing behavior.
SyntaxColoring Aug 13, 2021
b6d95af
Port ProtocolManager and SessionManager cleanup code.
SyntaxColoring Aug 13, 2021
3b854b2
Delete obsolete comment.
SyntaxColoring Aug 13, 2021
dc5032d
Handle `InitializationOngoingError` in `get_rpc_server()` (sort of).
SyntaxColoring Aug 13, 2021
b7ac395
Clean up some of `lifetime_dependencies`' imports.
SyntaxColoring Aug 13, 2021
8827b2f
Delete unnecessary todo comment.
SyntaxColoring Aug 13, 2021
3573b80
Don't unnecessarily wrap `motion_lock` in a context manager.
SyntaxColoring Aug 13, 2021
59d46f3
Clarify a comment.
SyntaxColoring Aug 13, 2021
615e06a
Remove now-unnecessary `@call_once` decorators.
SyntaxColoring Aug 13, 2021
8d06237
Remove print debug statements.
SyntaxColoring Aug 14, 2021
abd5d19
Remove unused import.
SyntaxColoring Aug 14, 2021
72a47dd
Remove resolved todo.
SyntaxColoring Aug 16, 2021
1c8d3b9
Rename `lifetime_dependency_set` args to `lifetime_dependencies`.
SyntaxColoring Aug 17, 2021
84a6ff6
Add some docstrings to `lifetime_dependencies` internal functions.
SyntaxColoring Aug 17, 2021
a960dd1
Make small improvements to how we store dependencies on `app`.
SyntaxColoring Aug 18, 2021
b3593da
Fix startup accidentally waiting for `ThreadManager` to be ready.
SyntaxColoring Aug 18, 2021
233ef3d
Reformat.
SyntaxColoring Aug 18, 2021
5343b36
Make `API.register_callback()` async again.
SyntaxColoring Aug 18, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion api/src/opentrons/hardware_control/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,7 @@ def _calculate_valid_attitude(self) -> DeckTransformState:
return rb_cal.validate_attitude_deck_calibration(
self._robot_calibration.deck_calibration)

async def register_callback(self, cb):
def register_callback(self, cb):
SyntaxColoring marked this conversation as resolved.
Show resolved Hide resolved
""" Allows the caller to register a callback, and returns a closure
that can be used to unregister the provided callback
"""
Expand Down
26 changes: 23 additions & 3 deletions robot-server/robot_server/app.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
"""Main FastAPI application."""


from functools import partial
import logging

from opentrons import __version__
Expand All @@ -8,20 +11,36 @@
from starlette.requests import Request
from starlette.middleware.base import RequestResponseEndpoint

from notify_server.clients import publisher as notify_server_publisher
from notify_server.settings import Settings as NotifyServerSettings
SyntaxColoring marked this conversation as resolved.
Show resolved Hide resolved


from .service.dependencies import (
get_rpc_server,
get_protocol_manager,
get_hardware_wrapper,
get_session_manager,
)
SyntaxColoring marked this conversation as resolved.
Show resolved Hide resolved

from .errors import exception_handlers
from .router import router
from .service import initialize_logging
from . import constants
from . import slow_initializing
from . import hardware_initialization

log = logging.getLogger(__name__)

def _make_event_publisher() -> notify_server_publisher.Publisher:
"""Create a notify-server event publisher instance."""
notify_server_settings = NotifyServerSettings()
event_publisher = notify_server_publisher.create(
notify_server_settings.publisher_address.connection_string()
)
# fixme(mm, 2021-08-09): Publisher has a .close() method that we must call, or
# we'll leak a socket.
SyntaxColoring marked this conversation as resolved.
Show resolved Hide resolved
return event_publisher


log = logging.getLogger(__name__)

app = FastAPI(
title="Opentrons OT-2 HTTP API Spec",
Expand Down Expand Up @@ -60,7 +79,8 @@ async def on_startup() -> None:
"""Handle app startup."""
initialize_logging()
# Initialize api
(await get_hardware_wrapper()).async_initialize()
factory = partial(hardware_initialization.initialize, _make_event_publisher())
app.state.hardware = slow_initializing.start_initializing(factory)
SyntaxColoring marked this conversation as resolved.
Show resolved Hide resolved


@app.on_event("shutdown")
Expand Down
62 changes: 62 additions & 0 deletions robot-server/robot_server/hardware_initialization.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
"""Utilities for initializing the hardware interface."""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Utilities" is always a bit of a warning sign for me. Something you need utilities, but if we have a hardware connection we need to initialize and hand out as a dependency, shouldn't those all be in the same place?

Is there a way we can restructure this PR to have robot_server/hardware.py be the thing that actually exports a startup / shutdown handler for the hardware along with the get_hardware dependency function? I'm imagining a system where any given dependency is able to use classes / functions in lifetime_dependencies to declare themselves a lifetime dependency.

For example, if this module was to export 2 or 3 methods, I think everything would be a lot easier to follow:

  1. def initialize_hardware(app_state: State) -> None
    • Creates a backround task to initialize a ThreadManager
    • Uses slow_initializing and/or lifetime_dependencies to attach it to app state
    • (Maybe) registers cleanup with lifetime_dependencies
  2. def get_hardware(app_state: State) -> ThreadManager
    • Retrieve the slow initializing hardware
    • Raise if not initialized
  3. (Maybe) def shutdown_hardware(app_state: State) -> None
    • Shut down the hardware
    • The necessity of this would depend on if lifetime_dependencies centralized cleanup or not



import logging
from pathlib import Path

from opentrons import ThreadManager, initialize as initialize_api
from opentrons.hardware_control.simulator_setup import load_simulator
from opentrons.hardware_control.types import HardwareEvent, HardwareEventType
from opentrons.util.helpers import utc_now

from notify_server.clients.publisher import Publisher
from notify_server.models import event, topics
from notify_server.models.hardware_event import DoorStatePayload

from .settings import get_settings


log = logging.getLogger(__name__)


async def initialize(event_publisher: Publisher) -> ThreadManager:
"""Initialize the API."""
app_settings = get_settings()
thread_manager: ThreadManager
if app_settings.simulator_configuration_file_path:
log.info(
f"Loading simulator from "
f"{app_settings.simulator_configuration_file_path}"
)
# A path to a simulation configuration is defined. Let's use it.
thread_manager = ThreadManager(
load_simulator, Path(app_settings.simulator_configuration_file_path)
)
else:
# Create the hardware
thread_manager = await initialize_api()
await _init_event_watchers(thread_manager, event_publisher)
log.info("Opentrons API initialized")
return thread_manager


async def _init_event_watchers(
thread_manager: ThreadManager,
event_publisher: Publisher,
) -> None:
"""Register the publisher callbacks with the hw thread manager."""
log.info("Starting hardware-event-notify publisher")

def publish_hardware_event(hw_event: HardwareEvent) -> None:
if hw_event.event == HardwareEventType.DOOR_SWITCH_CHANGE:
payload = DoorStatePayload(state=hw_event.new_state)
else:
return
topic = topics.RobotEventTopics.HARDWARE_EVENTS
publisher = "robot_server_event_publisher"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ The publisher value changed because it was previously based on the name of the containing class, which no longer exists.

event_to_publish = event.Event(
createdOn=utc_now(), publisher=publisher, data=payload
)
event_publisher.send_nowait(topic, event_to_publish)

thread_manager.register_callback(publish_hardware_event)
86 changes: 0 additions & 86 deletions robot-server/robot_server/hardware_wrapper.py

This file was deleted.

56 changes: 18 additions & 38 deletions robot-server/robot_server/service/dependencies.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,11 @@

Copy link
Contributor Author

@SyntaxColoring SyntaxColoring Aug 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ This diff got weird. I suggest viewing this file in split mode.

from opentrons.hardware_control import ThreadManager, ThreadedAsyncLock

from robot_server import constants, util, errors
from robot_server.hardware_wrapper import HardwareWrapper
from robot_server import app, constants, util, errors
SyntaxColoring marked this conversation as resolved.
Show resolved Hide resolved
from robot_server.service.session.manager import SessionManager
from robot_server.service.protocol.manager import ProtocolManager
from robot_server.service.legacy.rpc import RPCServer

from notify_server.clients import publisher
from notify_server.settings import Settings as NotifyServerSettings
from robot_server.slow_initializing import InitializationOngoingError, SlowInitializing


class OutdatedApiVersionResponse(errors.ErrorDetails):
Expand All @@ -32,38 +29,30 @@ class OutdatedApiVersionResponse(errors.ErrorDetails):
)


@util.call_once
async def get_event_publisher() -> publisher.Publisher:
"""A dependency creating a single notify-server event
publisher instance."""
notify_server_settings = NotifyServerSettings()
event_publisher = publisher.create(
notify_server_settings.publisher_address.connection_string()
)
return event_publisher
async def get_app_state() -> State:
"""Get the Starlette application's state from the framework.

See https://www.starlette.io/applications/#storing-state-on-the-app-instance
for more details.
"""

@util.call_once
async def get_hardware_wrapper(
event_publisher: publisher.Publisher = Depends(get_event_publisher),
) -> HardwareWrapper:
"""Get the single HardwareWrapper instance."""
return HardwareWrapper(event_publisher=event_publisher)
# Ideally, we would access the app state through request.app.state. However,
# this function might be depended upon by a WebSocket endpoint, and current FastAPI
# (v0.54.1) raises runtime errors when trying to resolve the built-in `request`
# dependency for WebSocket endpoints.
return app.app.state


async def get_hardware(
api_wrapper: HardwareWrapper = Depends(get_hardware_wrapper),
) -> ThreadManager:
async def get_hardware(state: State = Depends(get_app_state)) -> ThreadManager:
"""Hardware dependency"""
hardware = api_wrapper.get_hardware()

if hardware is None:
slow_initializing_hardware: SlowInitializing[ThreadManager] = state.hardware
try:
return slow_initializing_hardware.get_if_ready()
except InitializationOngoingError:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail="Robot motor controller is not ready.",
)

return hardware
) from None


@util.call_once
Expand Down Expand Up @@ -132,15 +121,6 @@ async def check_version_header(
request.state.api_version = min(requested_version, constants.API_VERSION)


def get_app_state(request: Request) -> State:
"""Get the Starlette application's state from the framework.

See https://www.starlette.io/applications/#storing-state-on-the-app-instance
for more details.
"""
return request.app.state


def get_unique_id() -> str:
"""Get a unique ID string to use as a resource identifier."""
return str(uuid4())
Expand Down
Loading