Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

allow workflow retention by number of workflows, rather than time #5369

Closed
alexec opened this issue Mar 11, 2021 · 5 comments · Fixed by #6854
Closed

allow workflow retention by number of workflows, rather than time #5369

alexec opened this issue Mar 11, 2021 · 5 comments · Fixed by #6854
Assignees
@NikeNano
Labels
area/controller Controller issues, panics area/gc Garbage collection, such as TTLs, retentionPolicy, delays, and more type/feature Feature request
Projects
Run The Business (incl. bugs)

Comments

@alexec
Copy link
Contributor

alexec commented Mar 11, 2021 •
edited
Loading

We want to limit the size of the data set. Rather than TTL based on time, what about TTL based on the number of workflows?

apiVersion: v1
kind: ConfigMap
data: 
  retentionPolicy: |
    completed: 10
    error: 3
    failed: 3

Questions:

  • Do we need to differentiate error vs failed vs completed? Is completed enough (frugality).
  • Is this independent from TTL? I think so, you define what is retained, not what is deleted. Just because something has not met its TTL does not mean we should retain it.
  • Is this useful at a cluster-level (easy to implement), at a workflow-level (hard to implement). (frugality)
@alexec alexec added the type/feature Feature request label Mar 11, 2021
@alexec alexec mentioned this issue Mar 11, 2021
Open
alexec added a commit to alexec/argo-workflows that referenced this issue Mar 12, 2021
@alexec
feat(controller): Workflow retention PoC. Closes argoproj#5369
3006b73 
Signed-off-by: Alex Collins <[email protected]>
@alexec alexec mentioned this issue Mar 12, 2021
Closed
@basanthjenuhb
Copy link
Contributor

Will be useful to have the have this number at a namespace level also.

@alexec
Copy link
Contributor Author

alexec commented Sep 26, 2021

@basanthjenuhb would you be interesting in implementing this feature?

@NikeNano
Copy link
Contributor

I would be happy to help out working on this @alexec.

@NikeNano NikeNano mentioned this issue Oct 3, 2021
Merged
@alexec alexec added this to To do in Run The Business (incl. bugs) via automation Oct 15, 2021
@NikeNano
Copy link
Contributor

Will updated the draft during the weekend.

@alexec alexec moved this from To do to In progress in Run The Business (incl. bugs) Oct 18, 2021
NikeNano added a commit to NikeNano/argo-workflows that referenced this issue Nov 29, 2021
@NikeNano
feat: added retention controller. Fixes argoproj#5369
0f6e84c 
Signed-off-by: NikeNano <[email protected]>
alexec pushed a commit that referenced this issue Dec 13, 2021
@NikeNano
feat: added retention controller. Fixes #5369 (#6854)
8d552fb 
Signed-off-by: NikeNano <[email protected]>
@alexec alexec moved this from In progress to Done in Run The Business (incl. bugs) Dec 13, 2021
@sarabala1979 sarabala1979 mentioned this issue Mar 1, 2022
Closed
@jixianyihao
Copy link

this retentionPolicy does not have any doc at all? if it can be used in production?

@agilgur5 agilgur5 mentioned this issue Nov 7, 2023
Open
3 tasks
@agilgur5 agilgur5 added area/controller Controller issues, panics area/gc Garbage collection, such as TTLs, retentionPolicy, delays, and more labels Feb 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@NikeNano NikeNano

Labels
area/controller Controller issues, panics area/gc Garbage collection, such as TTLs, retentionPolicy, delays, and more type/feature Feature request
Projects
Run The Business (incl. bugs)
Done
Milestone
No milestone
5 participants
@alexec @agilgur5 @jixianyihao @basanthjenuhb @NikeNano