Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DA] Update default AutomationConditionSensorDefinition name #23017

Conversation

OwenKephart
Copy link
Contributor

Summary & Motivation

Creates a one-time migration to handle the fact that we're updating the default sensor name. In these cases, we want to make sure that we preserve the history of these default sensors (particularly their cursors) so that this change is not disruptive.

Still working on creating tests.

How I Tested These Changes

Copy link
Contributor Author

OwenKephart commented Jul 15, 2024

@OwenKephart OwenKephart force-pushed the 07-15-Update_default_AutomationConditionSensorDefinition_name branch 3 times, most recently from 347e19c to 0f5b4ad Compare July 16, 2024 19:25
@OwenKephart OwenKephart changed the title Update default AutomationConditionSensorDefinition name [DA] Update default AutomationConditionSensorDefinition name Jul 16, 2024
@OwenKephart OwenKephart marked this pull request as ready for review July 16, 2024 19:25
@OwenKephart OwenKephart force-pushed the 07-15-Update_default_AutomationConditionSensorDefinition_name branch from 0f5b4ad to 3ed5d7c Compare July 16, 2024 20:09
@OwenKephart OwenKephart force-pushed the 07-08-Rename_AutoMaterializeSensorDefinition branch from 26454bb to 60844d5 Compare July 17, 2024 16:42
@OwenKephart OwenKephart force-pushed the 07-15-Update_default_AutomationConditionSensorDefinition_name branch from 3ed5d7c to b5bbe50 Compare July 17, 2024 16:42
@@ -626,6 +653,43 @@ def _create_initial_sensor_cursors_from_raw_cursor(

return result

def _rename_auto_materialize_states(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is more of a copy than a rename right? (which is good in the hopefully unlikely case that a user needs to roll back - the old state with the old name will still be there, harmless)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's correct yeah -- changed the comment here to be a bit more accurate and updated the name to _copy_default_auto_materialize_sensor_states.

@@ -509,8 +524,20 @@ def _run_iteration_impl(
)

set_has_migrated_to_sensors(instance)
if not get_has_migrated_sensor_names(instance):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just confirming my understanding - for new users who have never used AMP before or had a sensor before, the only thing this does is set a key in the kvs table right? Nothing gets moved or migrated in that case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's correct yep -- it will not find any historical instigator data in the database and so no migration is actually happening

Comment on lines 673 to 680
for sensor, repo in sensors_and_repos:
# find the corresponding sensor with the updated name
if (
sensor.name == "default_automation_condition_sensor"
and repo.get_external_origin() == instigator_state_repository_origin
):
new_auto_materialize_state = InstigatorState(
sensor.get_external_origin(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i am a little unsure why we need sensors_and_repos here at all - can the new origin to use not always be derived from the old origin?

i.e. could this not be

new_sensor_origin = instigator_state.origin._replace(instigator_name="default_automation_condition_sensor")

and then we create it, whether or not there is a sensor with that name?

Not sure if it matters in practice but seems a bit simpler

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah the main thing here was just having access to the selector id, which given your comment below is actually possible to get without the sensor itself. This seems a lot better, and guards against a case where the given code location is unavailable at the time of the migration

instigator_state.status,
instigator_state.instigator_data,
)
result[sensor.selector_id] = new_auto_materialize_state
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you took out sensors_and_repos this would then become

result[new_sensor_origin.get_selector().get_id()]

@OwenKephart OwenKephart force-pushed the 07-15-Update_default_AutomationConditionSensorDefinition_name branch 2 times, most recently from 24c9ebd to c5fbcc3 Compare July 17, 2024 21:53
@OwenKephart OwenKephart force-pushed the 07-08-Rename_AutoMaterializeSensorDefinition branch from 60844d5 to 3e6b391 Compare July 19, 2024 16:38
@OwenKephart OwenKephart force-pushed the 07-15-Update_default_AutomationConditionSensorDefinition_name branch from c5fbcc3 to 35e2fec Compare July 19, 2024 16:39
@graphite-app graphite-app bot added the area: docs Related to documentation in general label Jul 19, 2024
Copy link
Contributor Author

OwenKephart commented Jul 19, 2024

Merge activity

  • Jul 19, 1:14 PM EDT: @OwenKephart started a stack merge that includes this pull request via Graphite.
  • Jul 19, 1:17 PM EDT: Graphite rebased this pull request as part of a merge.
  • Jul 19, 1:18 PM EDT: @OwenKephart merged this pull request with Graphite.

@OwenKephart OwenKephart force-pushed the 07-08-Rename_AutoMaterializeSensorDefinition branch from 3e6b391 to 2432442 Compare July 19, 2024 17:15
Base automatically changed from 07-08-Rename_AutoMaterializeSensorDefinition to master July 19, 2024 17:16
@OwenKephart OwenKephart force-pushed the 07-15-Update_default_AutomationConditionSensorDefinition_name branch from 35e2fec to d6792f7 Compare July 19, 2024 17:16
@OwenKephart OwenKephart merged commit e75e2dd into master Jul 19, 2024
1 check was pending
@OwenKephart OwenKephart deleted the 07-15-Update_default_AutomationConditionSensorDefinition_name branch July 19, 2024 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: docs Related to documentation in general
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants