You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When a user develops models in their Quary project, builds the project (quary build) to materialize those models in the target schema/database, but later deletes some of those model definitions from the project, the corresponding objects remain in the target schema even though they are no longer part of the current project. Over time, this can lead to the accumulation of stale, unused objects in the target schema, increasing cloud storage costs and cluttering the data warehouse.
Quary currently lacks an easy way for users to clean up these orphaned objects and keep their target schema in sync with the current state of their project.
Suggested solution
Implement a new Quary operation, tentatively called quary clean, that will:
Introspect the current Quary project to determine what models/objects it defines
Compare this to the objects actually present in the target schema
Identify any objects in the target that no longer map to any models in the current project
Provide the user clear output listing out these stale objects
Optionally, allow the user to specify a flag that will generate the DDL statements to drop those stale objects
Optionally, allow the user to specify an additional flag to actually execute those drop statements
This will allow users to manually trigger clean-up of their target schemas and remove any models that are no longer part of their current project.
As a potential incremental step towards this goal, we could first implement a simpler quary diff operation that just performs steps 1-4 above and outputs the diff between project and target, without handling the dropping of stale objects. This would still provide visibility to users about stale objects.
The text was updated successfully, but these errors were encountered:
Problem to be solved
When a user develops models in their Quary project, builds the project (
quary build
) to materialize those models in the target schema/database, but later deletes some of those model definitions from the project, the corresponding objects remain in the target schema even though they are no longer part of the current project. Over time, this can lead to the accumulation of stale, unused objects in the target schema, increasing cloud storage costs and cluttering the data warehouse.Quary currently lacks an easy way for users to clean up these orphaned objects and keep their target schema in sync with the current state of their project.
Suggested solution
Implement a new Quary operation, tentatively called
quary clean
, that will:This will allow users to manually trigger clean-up of their target schemas and remove any models that are no longer part of their current project.
As a potential incremental step towards this goal, we could first implement a simpler
quary diff
operation that just performs steps 1-4 above and outputs the diff between project and target, without handling the dropping of stale objects. This would still provide visibility to users about stale objects.The text was updated successfully, but these errors were encountered: