Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore internal release 1.5.1 #15303

Closed
wants to merge 1,030 commits into from
Closed

Conversation

sfoster1
Copy link
Member

Release tracking branch for internal release 1.5.1, which will contain updated tip overlap values for all PD protocols and all python protocols using API level >= 2.19.

jerader and others added 30 commits April 8, 2024 16:15
Closes AUTH-255

# Overview

Adds a POST method to the existing `/protocols/{protocolId}/analyses`
endpoint in order to post a new analysis for an existing protocol.

This endpoint will take a request body with two optional fields:
- `runTimeParameterValues`
- `forceReAnalyze`

The new method can affect the analyses in three ways:
1. When the request is sent with `forceReAnalyze=True`, the server will
unconditionally start a new analysis for the protocol using any RTP data
sent along with it. It will return a 201 CREATED status and respond with
a list of analysis summaries of all the analyses (ordered oldest first),
including the newly started analysis.
2. When the request is sent without the `forceReAnalyze` field (or with
`forceReAnalyze=False`), then the server will check the last analysis of
the protocol
- if the RTP values used for it were **different** from the RTP values
sent with the current request, then the server will start a new analysis
using the new RTP values. It will return a 201 CREATED status and
respond with a list of analysis summaries of all the analyses, including
the newly started analysis.
- if the RTP values used for it were the **same** as the RTP values sent
with the current request, then the server will **NOT** start a new
analysis. It will return a 200 OK status, and simply return the existing
list of analysis summaries.

This request requires the last analysis of the protocol to have been
completed before handling this request. If the last analysis is pending,
it will return a 503 error.

# Test Plan

Test out the above three cases and anything else you can think might
affect the behavior.
It's pretty well tested in unit & integration tests and it is also the
same logic used for handling analyses from `POST /protocols`, so it is
expected to work well when used as tested in integration tests.

# Review requests

Usual review for code sanity check.

# Risk assessment

Low. New HTTP API not yet used anywhere.
<!--
Thanks for taking the time to open a pull request! Please make sure
you've read the "Opening Pull Requests" section of our Contributing
Guide:


https://github.com/Opentrons/opentrons/blob/edge/CONTRIBUTING.md#opening-pull-requests

To ensure your code is reviewed quickly and thoroughly, please fill out
the sections below to the best of your ability!
-->

# Overview

This PR adds a new testing script that allows us to test all kinds of
variations of the liquid-sense routine
it adds some additional features in the hardware control layer to change
up output options to during the probe so we can gate using the
buffer-on-pipette feature to a firmware version flag, since that feature
has to be compiled in separately

<!--
Use this section to describe your pull-request at a high level. If the
PR addresses any open issues, please tag the issues here.
-->

# Test Plan

<!--
Use this section to describe the steps that you took to test your Pull
Request.
If you did not perform any testing provide justification why.

OT-3 Developers: You should default to testing on actual physical
hardware.
Once again, if you did not perform testing against hardware, justify
why.

Note: It can be helpful to write a test plan before doing development

Example Test Plan (HTTP API Change)

- Verified that new optional argument `dance-party` causes the robot to
flash its lights, move the pipettes,
then home.
- Verified that when you omit the `dance-party` option the robot homes
normally
- Added protocol that uses `dance-party` argument to G-Code Testing
Suite
- Ran protocol that did not use `dance-party` argument and everything
was successful
- Added unit tests to validate that changes to pydantic model are
correct

-->

# Changelog

<!--
List out the changes to the code in this PR. Please try your best to
categorize your changes and describe what has changed and why.

Example changelog:
- Fixed app crash when trying to calibrate an illegal pipette
- Added state to API to track pipette usage
- Updated API docs to mention only two pipettes are supported

IMPORTANT: MAKE SURE ANY BREAKING CHANGES ARE PROPERLY COMMUNICATED
-->

# Review requests

<!--
Describe any requests for your reviewers here.
-->

# Risk assessment

<!--
Carefully go over your pull request and look at the other parts of the
codebase it may affect. Look for the possibility, even if you think it's
small, that your change may affect some other part of the system - for
instance, changing return tip behavior in protocol may also change the
behavior of labware calibration.

Identify the other parts of the system your codebase may affect, so that
in addition to your own review and testing, other people who may not
have the system internalized as much as you can focus their attention
and testing there.
-->

---------

Co-authored-by: caila-marashaj <[email protected]>
<!--
Thanks for taking the time to open a pull request! Please make sure
you've read the "Opening Pull Requests" section of our Contributing
Guide:


https://github.com/Opentrons/opentrons/blob/edge/CONTRIBUTING.md#opening-pull-requests

To ensure your code is reviewed quickly and thoroughly, please fill out
the sections below to the best of your ability!
-->

# Overview

<!--
Use this section to describe your pull-request at a high level. If the
PR addresses any open issues, please tag the issues here.
-->

Improved ABR Error Data Collection

# Test Plan

Tested code on multiple robots.

# Changelog

Added function to download robot logs
Added lines of code to move error documents (run log, calibration log,
robot logs) into folder named after ticket.
Adds robot run to ABR sheet and links JIRA ticket
Added extra lines to abr_scale to read scale more often
Edited ABR calibration script to ensure duplicate calibrations are not
added.

# Review requests

Is 5000 lines of recording enough to capture robot error if script is
run immediately?
Is there any manipulation to robot logs that can be down to make error
analysis more efficient.

# Risk assessment

<!--
Carefully go over your pull request and look at the other parts of the
codebase it may affect. Look for the possibility, even if you think it's
small, that your change may affect some other part of the system - for
instance, changing return tip behavior in protocol may also change the
behavior of labware calibration.

Identify the other parts of the system your codebase may affect, so that
in addition to your own review and testing, other people who may not
have the system internalized as much as you can focus their attention
and testing there.
-->
Into edge instead of release branch.

Was #14840

Co-authored-by: Jamey Huffnagle <[email protected]>
* fix(shared-data, app): fix runtime parameters range display
When we switched to vite, we had to switch all the stuff we'd been
injecting at pack time via webpack environment/define plugins to vite's
`define` config functionality. The biggest thing we specify that way is
the app version, which is used across the stack for display and for
logic.

In the commit that switched to vite, we added that injection for the
app-shell vite configs but did not add it for the app vite configs. That
meant that at runtime, the version value was undefined, which breaks
robot update notifications and causes the app version in the general
settings tab to not display (it also makes the logo wrong on internal
releases but that's a bit less important).

The fix is to inject the version into the app build again. This is made
a little more complicated because if you're doing stuff to the app vite
config, it has to work in both the vite devserver and the vite offline
packaging environments, and the vite devserver doesn't allow commonjs,
and the git-version script that gives us the version is commonjs. For
the purposes of vite's devserver, "doesn't work with cjs" actually just
means "doesn't support require()", so you can use a hybrid syntax that
uses import-statements but still module.export instead of export
statements.

Unfortunately, the git-version script is also used in the
electron-builder config for the app-shell and the app-shell-odd, and the
electron-builder config is run via node, and to import an ESM from a
node CJS script - which electron-builder.config.js is - you need to
change your import syntax to dynamic import and you need to make the
import target explicitly (to node) an ESM, aka change its extension, and
you need to use full ESM syntax including exports. This also goes for
the create-release script.

So that means that
- git-version.js becomes git-version.mjs and uses full ESM syntax
- that means that everywhere it's imported we need to import it by full
path with extension instead of module name
- also we need to import it dynamically in the electron config
- oh and we need to actually add the define configs so we get the
version in the app

And then finally we show the version again.

Also, remove some old webpack.config.js files that aren't used anymore.

Closes EXEC-385
* fix(shared-data, app): fix small issues in app
* fix(discovery-client): fix import statement
…rons-ai (#14788)

* feat(opentrons-ai-client, opentrons-ai-server): add folders for opentrons-ai
This PR is an automated snapshot update request. Please review the
changes and merge if they are acceptable or find you bug and fix it.

Co-authored-by: y3rsh <[email protected]>
* fix(app): fix rtp slideout issue
* fix(app-shell-odd): fix typo in vite-config
The schema changes in edge weren't in release and need to be manually
merged.

Closes RQA-2558
<!--
Thanks for taking the time to open a pull request! Please make sure
you've read the "Opening Pull Requests" section of our Contributing
Guide:


https://github.com/Opentrons/opentrons/blob/edge/CONTRIBUTING.md#opening-pull-requests

To ensure your code is reviewed quickly and thoroughly, please fill out
the sections below to the best of your ability!
-->

# Overview

Calculates module ramp rates based on run log and uploads to google
sheet.

# Test Plan

Ramp rate script tested on all three modules with different robots. 

# Changelog

Created module ramp rate script to find ramp rate runs in run log folder
and upload ramp rates to abr-run-data sheet
Also changed IP address in error recording to a user input rather than
an input in order to allow the command to be created into a desktop
shortcut.

# Review requests

<!--
Describe any requests for your reviewers here.
-->

# Risk assessment
* fix(app): remove unnecessary console.log
ncdiehl11 and others added 18 commits May 10, 2024 15:52
 RQA-2700
Ensure automatic tip tracking for partial configurations does not exceed the limits of the tiprack it is iterating over
<!--
Thanks for taking the time to open a pull request! Please make sure
you've read the "Opening Pull Requests" section of our Contributing
Guide:


https://github.com/Opentrons/opentrons/blob/edge/CONTRIBUTING.md#opening-pull-requests

To ensure your code is reviewed quickly and thoroughly, please fill out
the sections below to the best of your ability!
-->

# Overview
After conducting hardware testing, we've formulated updated tip overlap
values for every tip size and pipette model. These adjustments apply to
1-channel, 8-channel, and 96-channel pipettes. These revised values aim
to mitigate overpressure occurrences in ABR.


[https://docs.google.com/spreadsheets/d/16dEmqulO4SIKU6E0aYTfVQEnZRTn3MwNF_WB8K1KmRM/edit#gid=0](url)


<!--
Use this section to describe your pull-request at a high level. If the
PR addresses any open issues, please tag the issues here.
-->

# Test Plan
None
<!--
Use this section to describe the steps that you took to test your Pull
Request.
If you did not perform any testing provide justification why.

OT-3 Developers: You should default to testing on actual physical
hardware.
Once again, if you did not perform testing against hardware, justify
why.

Note: It can be helpful to write a test plan before doing development

Example Test Plan (HTTP API Change)

- Verified that new optional argument `dance-party` causes the robot to
flash its lights, move the pipettes,
then home.
- Verified that when you omit the `dance-party` option the robot homes
normally
- Added protocol that uses `dance-party` argument to G-Code Testing
Suite
- Ran protocol that did not use `dance-party` argument and everything
was successful
- Added unit tests to validate that changes to pydantic model are
correct

-->

# Changelog
### **Pipette shared data**
### P1KS Single Channel
- Updated V3_5 Tip Overlap values for 50uL, 200uL, and 1000uL tips.
- Updated V3_6 Tip Overlap values for 50uL 200uL, and 1000uL tips.
### P50S Single Channel Pipette
- Updated V3_5 Tip Overlap values for 50uL, 200uL, and 1000uL tips.
- Updated V3_6 Tip Overlap values for 50uL 200uL, and 1000uL tips.
### P1KM 8 Channel Pipette
- Updated V3_5 Tip Overlap values for 50uL, 200uL, and 1000uL tips.
- Updated V3_6 Tip Overlap values for 50uL 200uL, and 1000uL tips.
### P50M 8 Channel Pipette
- Updated V3_5 Tip Overlap values for 50uL, 200uL, and 1000uL tips.
- Updated V3_6 Tip Overlap values for 50uL 200uL, and 1000uL tips
### P1H 95-Channel Pipette
- Updated V3_5 Tip Overlap values for 50uL, 200uL, and 1000uL tips.
- Updated V3_6 Tip Overlap values for 50uL 200uL, and 1000uL tips

<!--
List out the changes to the code in this PR. Please try your best to
categorize your changes and describe what has changed and why.

Example changelog:
- Fixed app crash when trying to calibrate an illegal pipette
- Added state to API to track pipette usage
- Updated API docs to mention only two pipettes are supported

IMPORTANT: MAKE SURE ANY BREAKING CHANGES ARE PROPERLY COMMUNICATED
-->

# Review requests

<!--
Describe any requests for your reviewers here.
-->

# Risk assessment
Low
<!--
Carefully go over your pull request and look at the other parts of the
codebase it may affect. Look for the possibility, even if you think it's
small, that your change may affect some other part of the system - for
instance, changing return tip behavior in protocol may also change the
behavior of labware calibration.

Identify the other parts of the system your codebase may affect, so that
in addition to your own review and testing, other people who may not
have the system internalized as much as you can focus their attention
and testing there.
-->

---------

Co-authored-by: Seth Foster <[email protected]>
#15178)

…(#15147)"

This reverts commit 0096c12.

The above commit needs to be gated behind a protocol API version.
# Overview

Communicating https://opentrons.atlassian.net/browse/RQA-2554 as a known
issue that will be present in 7.3.0

# Review requests

Accurate and sufficient?

# Risk assessment

none
…start (#15179)

Show a 'Run was never started' InfoScreen under RunPreview for a terminal run that was never started (has 0 commands). Ensure we only fetch once to eliminate flickering.

Closes RQA-2717
…eAllCommandsAsPreSerializedList (#15188)

React query does not properly handle objects passed in useQuery's
queryKey array argument that contain keys with `undefined` values. Here,
in `useAllCommandsAsPreSerializedList`, I map undefined values to null
so that they are properly cached and do not trigger a refetch when the
host object does not change.
…ype via RTP (#15194)

# Overview

Letting people know they can get chicken-and-egg'd if they apply labware
offsets, set RTPs that change their labware, and then reanalysis gets
rid of their original offsets.

# Review requests

Is this clear? This is kind of a tricky one to wrap up in a single
sentence.

# Risk assessment

zip
…w OT2 module setup (#15204)

Don't look to deck configuration for matching specced modules to
attached modules within the module
setup step of the OT-2

Closes [RQA-2735](https://opentrons.atlassian.net/browse/RQA-2735)
…15205)

Closes RQA-2731

On OT-2, after a pipette is attached or detached successfully, the exit
button should only be clickable once, so that many home commands are not
emitted in rapid succession with each click.
Closes RQA-2730

Remove InterventionModal if run is finishing
…15202)

Instead of the previous method of staging a batch of changes to deck
configuration before commiting
them with the confirm button, to avoid synchronization issues, make the
ODD deck configurator behave
like the desktop version. all updates immediately permeate to the
robot's source of truth

Re: https://opentrons.atlassian.net/browse/RQA-2669
I think this would be a subtle side effect of a previous pr trying to
improve PE end behavior.

The reason this fix is required is that when a client cancels a protocol
by stopping the engine, the `StopAction` sets the run result (good) and
doesn't set a run error (good, correct, there wasn't one, this is a
cancel). In 7.2, with the behavior this PR now restores, a
`FinishAction` that might contain an error wouldn't set that error in
the run if it already had a _result_, whether or not it had an _error_.
In 7.3, it would set the error if the engine had no error, which it
wouldn't, because there is no error when you stop.

## Testing
- [x] With this PR, cancelling a run on a real robot causes the ODD to
show an inactive error details button
- [x] With this PR, stopping a run by hitting an estop still causes the
ODD and desktop to display that an estop error cancelled the run

Closes RQA-2737

---------

Co-authored-by: Max Marrone <[email protected]>
…nt for thermocycler occupying 2 slots (#15239)

Ensure slot conflict checking accounts for the thermocycler being comprised of two fixtures in the deck config when validating the cutout fixtures.
…f the required modules (#15232)

Fix bug where we were accidentally filtering magnetic modules instead of
magnetic blocks from the
required modules list from analysis.

Closes [RQA-2754](https://opentrons.atlassian.net/browse/RQA-2754)
…t channels (#15259)

Closes RQA-2780
Ensures the engine correctly identifies the next valid column for an 8ch pickup after a
sequence of single channel pickups.
This carves out a 2.19 API version that will contain updated tip overlap
data.

Closes EXEC-452
@sfoster1 sfoster1 added the DO NOT MERGE Indicates a PR should not be merged, even if there's a shiny green merge button available label May 31, 2024
@sfoster1 sfoster1 requested review from a team as code owners May 31, 2024 16:43
@sfoster1 sfoster1 requested a review from a team May 31, 2024 16:43
@sfoster1 sfoster1 requested a review from a team as a code owner May 31, 2024 16:43
@sfoster1 sfoster1 requested review from shlokamin and removed request for a team May 31, 2024 16:43
sfoster1 and others added 3 commits June 4, 2024 14:36
…15323)

We need to have multiple versions of the tip overlap data for at least
flex pipettes so that we can ship updated overlap data without
inadvertently changing the behavior of protocols on stable API versions.
This PR accomplishes that.


# Details
## separate versioning
The data in shared-data has a version, but that version is separate from
versions in other parts of the stack. This is on purpose, because the
version has to be written in shared-data, and read in protocol engine. I
think it's gross to write API versions in shared data, but we've done it
before and could do it again; the real sticking point was the engine.
The engine, in modern protocols, loads the tip overlap and does all that
data management itself; it would need to select the appropriate version
from the dict. We really don't want API versions there.

Therefore, there's a separate simple versioning scheme that's a little
more like the alternate liquid handling function scheme that we haven't
used in years, but with an actual string key for the version instead of
implicit versioning by positioning in an array.

There will instead be a specific map of API versions to overlap versions
living inside the source of the engine core for the python protocol API
(that's the only place it needs to be - for why, read on).

## engine state management and data lifecycle
There were basically two ways to implement this. The first way is to
save all the data - save the full versioned tip overlap dictionary, and
save which version we've selected. Since I listed this first, you can
guess I didn't do this. Instead, what I did was add a version to the two
engine commands that load pipette data - `load_pipette` and
`configure_for_volume` - and then have the functions those commands call
pull the specified version out of the full config, which means that we
only have to save the specific version in state and state doesn't have
to change. This also means that we don't have to worry about data
storage migration and compatibility.

One huge note here: **the engine argument parsing gives you the MOST
RECENT tip overlap if you don't specify one**. That's in bold because
it's not the way we've implemented similar things. It's necessary
because we want PD protocols to get the new overlap values without
having to change anything.

## python api integration
The python API, however, really wants to keep the old values (until
you're on a certain version - we'll implement that when we actually add
the new values). We do that in a couple ways:
- the old pipette dictionary value that has the tip overlap dict is
still there, using "v0". That means that protocol API versions that use
that dict, which is all those not using the engine core, get the old
values consistently for free
- for protocol api versions that do use the engine core, for now we hard
specify v0.

## json protocol integration

New JSON protocols, any that directly create engine commands rather than
being mapped through python protocol API values, will get the new
overlaps because they won't specify the parameter to the load pipette or
configure for volume engine commands. Older JSON protocols that do adapt
the python protocol API commands will get the python behavior.

# Testing
- This PR gets regression tested - it doesn't really contain any new
features, because it doesn't really contain any new data.
- OT-2 protocol behavior should be adequately covered by the g-code
testing
  - OT-2 calibration should probably be run real quick
- Flex behavior should be adequately tested by running LPC and a
protocol - this applies to all protocols and can be easily tested.
- Flex calibration and LPC doesn't use tips and therefore isn't affected


Closes EXEC-451
We recently added an explicit runtime dependency on packaging but didn't
add it to our deps. Add it now.

Picked 22.0 as the version to depend on because it introduced type
stubs.

Closes RDEVOPS-98
#15147) (#15284)

This is #15147 again but this time outside the context of an extremely
imminent release.

While this code needs to be merged, we should carefully consider how we
do so. We can't ship this until we can separate this behavior by API
level so that customers who may have tuned in offsets based on the old
incorrect tip offsets won't have their protocols invalidated - this data
needs to be loaded by API level.

Closes EXEC-452

---------

Co-authored-by: Carlos-fernandez <[email protected]>
@sfoster1 sfoster1 closed this Jun 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DO NOT MERGE Indicates a PR should not be merged, even if there's a shiny green merge button available
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet