Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(robot-server): clear ChangeNotifier's internal event immediately after waiting #15103

Conversation

mjhuff
Copy link
Contributor

@mjhuff mjhuff commented May 6, 2024

Closes RQA-2677

Overview

ChangeNotifier is the interface for alerting robot-server of notification updates deriving from Protocol Engine. Protocol Engine notify()ies, and the PublisherNotifier wait()s. After the event flag is set() in notify(), the PublisherNotifier no longer waits (correctly). However, the event flag isn't cleared immediately on receiving a new event...it's cleared after PublisherNotifer fires all its registered callbacks. This causes some events to be dropped if Protocol Engine tries setting the flag before PublisherNotifier has gotten around to clearing it... woops.

The reason we haven't seen this problem earlier is because we currently don't have too many immediately back-to-back events firing with publishers that would be interested in those events, but in the specific scenario mentioned in the ticket, the following is happening:

  • A play action hits PE, and PE sets the event flag via notify().
  • PublisherNotifier starts firing through its registered callbacks.
  • The "command succeeded" & "next running command" actions hit PE very shortly after the play action, and the set event flag is set from True to True redundantly.
  • PublisherNotifier finishes its callbacks then sets the event flag to False...dropping these new events.

The fix: simply clear the event flag immediately after waiting.

Test Plan

  • You can test this with any protocol containing a manual move labware command + a command afterwards. Here's the protocol I used.
  • Verify that the Intervention Modal derenders on the ODD immediately after the command is completed (so like 1 second after pressing the button), NOT a whole command after the command proceeding the manual move command.

Changelog

  • Fixed a bug in which commands after manual move commands were slow to update on the ODD.

Risk assessment

low

@mjhuff mjhuff requested a review from a team May 6, 2024 20:41
@mjhuff mjhuff requested a review from a team as a code owner May 6, 2024 20:41
…ting

ChangeNotifier is the interface for alerting robot-server of notification updates deriving from Protocol Engine. Protocol Engine notify()ies, and the PublisherNotifier wait()s. After the event flag is set() in notify(), the PublisherNotifier no longer waits (correctly). However, the event flag isn't cleared immediately on receiving a new event...it's cleared after PublisherNotifer fires all its registered callbacks. This causes some events to be dropped if Protocol Engine tries setting the flag before PublisherNotifier has gotten around to clearing it... woops.

The reason we haven't seen this problem earlier is because we currently don't have too many immediately back-to-back events firing, but in the specific scenario mentioned in the ticket, the following is happening:

A play action hits PE, and PE sets the event flag via notify().
PublisherNotifier starts firing through its registered callbacks.
The command succeeded & next running command updates hit PE very shortly after the play action, and the set event flag is set from True to True redundantly.
PublisherNotifier finishes its callbacks then sets the event flag to False...dropping these new events.
The fix: simply clear the event flag immediately after waiting.
@mjhuff mjhuff force-pushed the robot_server-clear-change-notifier-event-immediately branch from 57ff88b to 1580fa2 Compare May 6, 2024 20:52
@mjhuff mjhuff requested a review from a team as a code owner May 6, 2024 20:52
@mjhuff mjhuff requested review from shlokamin and removed request for a team May 6, 2024 20:52
@mjhuff mjhuff changed the base branch from edge to chore_release-7.3.0 May 6, 2024 20:52
@mjhuff mjhuff removed the request for review from shlokamin May 6, 2024 20:52
@mjhuff mjhuff changed the title fix(api): clear ChangeNotifier's internal event immediately after waiting fix(robot-server): clear ChangeNotifier's internal event immediately after waiting May 7, 2024
Copy link
Member

@sfoster1 sfoster1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, good catch

@mjhuff mjhuff merged commit cfed247 into chore_release-7.3.0 May 7, 2024
23 checks passed
@mjhuff mjhuff deleted the robot_server-clear-change-notifier-event-immediately branch May 7, 2024 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants