Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(app-testing): only specify labware that is used when running from tests #15508

Conversation

DerekMaggio
Copy link
Contributor

@DerekMaggio DerekMaggio commented Jun 25, 2024

Overview

Fixes https://opentrons.atlassian.net/browse/EXEC-582

I ran into an issue when adding PL protocols to snapshot testing.
We were passing all custom labware in the labware folder to python -m opentrons.cli.analyze. Subsequently, when you add another custom labware, every protocol has the custom labware added to its command, meaning it ends up in the analysis. This causes every snapshot to need to be updated when you add a new custom labware.

This PR slightly modifies how we build the analyze command when we use generate_analyses_from_test. Instead of passing all of the custom labware, we only pass the custom labware that the Protocol object specifies. Doing this ensures that adding a new custom labware does not every protocol to change.

NOTE: The functionality of analyze_against_image is maintained as passing all the custom labware.

Test Plan

  • Verify bug
    • Run snapshots and make sure they all pass
    • Without any changes to the logic in generate_analyses.py, add new custom labware to the labware folder. The added custom labware should not be specified in any of the ran protocols
    • Run snapshot tests and verify that they all fail because new labware was added to analysis output
  • Remove previously added custom labware
  • Verify fix
    • Add changes to generate_analyses.py
    • Run snapshot tests.
    • Verify all tests fail ONLY because any extraneous labware has been removed.
    • Verify that any protocols that use custom labware still have them in their analyses
    • Update snapshots on branch
    • Add the same labware that you used to verify the bug
    • Rerun the snapshot tests against the new snapshots and confirm that all tests pass (are not changed by adding that labware)

Review requests

None

Risk assessment

Low

functionality of running from image is maintained
@DerekMaggio DerekMaggio requested review from y3rsh, SyntaxColoring and a team June 25, 2024 15:39
@DerekMaggio DerekMaggio self-assigned this Jun 25, 2024
@DerekMaggio DerekMaggio marked this pull request as ready for review June 25, 2024 15:39
@DerekMaggio DerekMaggio requested review from a team as code owners June 25, 2024 15:39
@DerekMaggio DerekMaggio changed the base branch from edge to EXEC-196-analysis-snapshot-improvements June 25, 2024 16:03
@DerekMaggio DerekMaggio merged commit 1cd71c0 into EXEC-196-analysis-snapshot-improvements Jun 26, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants