Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[run status sensors] scope run status sensor to code location #20587

Merged
merged 3 commits into from
Mar 20, 2024

Conversation

jamiedemaria
Copy link
Contributor

@jamiedemaria jamiedemaria commented Mar 20, 2024

Summary & Motivation

This PR fixes a bug in run status sensors where jobs in other code locations were erroneously ticking the run status sensor when it was configured to only tick for jobs in the current code location.

Run status sensors can be set to only monitor specific jobs in the code location/repository where the sensor is located. This functionality was added before Definitions was introduced and we used the repository name to ensure that only jobs in the current repository were considered. With code locations/definitions, now all the repositories have the same name, so jobs in other code locations can tick the sensor now. To solve this we pipe the CodeLocationOrigin to the sensor evaluation context so that we can compare the code location name for the sensor with the code location name for the job.

How I Tested These Changes

New unit test, I also run local host cloud and added two code locations with the same named job in each. Confirmed that the job in code location 2 didn't tick the sensor in code location 1

resolves #14353 and #13076

Copy link
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @jamiedemaria and the rest of your teammates on Graphite Graphite

@jamiedemaria jamiedemaria force-pushed the jamie/scope_run_status_sensors_to_code_location branch from 6509a72 to 58946e8 Compare March 20, 2024 15:52
@@ -402,6 +409,11 @@ def repository_def(self) -> Optional["RepositoryDefinition"]:
"""Optional[RepositoryDefinition]: The RepositoryDefinition that this sensor resides in."""
return self._repository_def

@property
def code_location_origin(self) -> Optional["CodeLocationOrigin"]:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think i'd prefer to keep this a non-public property for now. Otherwise, users may start to use it in the body of their sensor functions and then will need a way to make CodeLocationOrigins and pass them to build_sensor_context for testing their sensors

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think non-public makes sense.

@jamiedemaria jamiedemaria force-pushed the jamie/scope_run_status_sensors_to_code_location branch from fd900fe to 1fb290e Compare March 20, 2024 17:52
@jamiedemaria jamiedemaria marked this pull request as ready for review March 20, 2024 17:53
Copy link
Contributor

@yuhan yuhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you!

@@ -402,6 +409,11 @@ def repository_def(self) -> Optional["RepositoryDefinition"]:
"""Optional[RepositoryDefinition]: The RepositoryDefinition that this sensor resides in."""
return self._repository_def

@property
def code_location_origin(self) -> Optional["CodeLocationOrigin"]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think non-public makes sense.

@jamiedemaria jamiedemaria merged commit bc0a2c1 into master Mar 20, 2024
1 check passed
@jamiedemaria jamiedemaria deleted the jamie/scope_run_status_sensors_to_code_location branch March 20, 2024 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[dagster-msteams] make_teams_on_run_failure_sensor monitors all code locations
2 participants