-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow to skip the save post-step #489
Conversation
I really like this approach! It's helped to save more valuable seconds on my builds than other solutions. |
Thanks @domjtalbot. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your suggestion!
Due to the type issues with env
always being a string, I think we'd need this to be an input rather than an environment variable so that we can properly evaluate it.
Also this does not make the cache key option, so could you update your PR description to remove "Fixes #210"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for reviewing @joshmgross !
I think we'd need this to be an input rather than an environment variable so that we can properly evaluate it
post-if
is undocumented and despite my best efforts to make it evaluate inputs I couldn't - only env
was available to me.
Due to the type issues with env always being a string...
I did address this one though - so perhaps using env
would be acceptable now?
I see this PR has gone through a first review by @joshmgross already. Will it be merged in the near future, or should use cases needing to skip the save step use e.g. a fork of this action? Regarding using environment variable: I think this is a good choice, if my understanding is correct that previous steps in the job then can dynamically/during runtime instruct the Thanks 🙂 |
This looks wonderful; one question, if you don't mind. I have a very large cache (~750 MB), and when I have multiple PRs in progress, it appears that the 5GB cache gets filled up with those, and the cache from |
@joshmgross Can you please re-review/merge this? I'd like to use it and would prefer not to rely on a fork. |
👋 Hey @zwass, the @actions/artifacts-actions team should be able to help out here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TY for proposing this change. Just some minor comments on the README changes, but otherwise looks good. Please ping me when ready and I can 🚢 this.
@@ -67,6 +70,8 @@ jobs: | |||
- uses: actions/checkout@v2 | |||
|
|||
- name: Cache Primes | |||
env: | |||
CACHE_SKIP_SAVE: true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given that this is the default example and skipping cache save is a special case, I wouldn't include this here. Perhaps move this into the Examples.md file and we can add some specific examples of where this is useful.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Something we can also highlight in the example is how users can either set this on the cache step as shown here or using echo "CACHE_SKIP_SAVE=true" >> $GITHUB_ENV
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the changes you've suggested on the readme file capture it well enough
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@roni-frantchi & @dhadka, would be great if this PR could be merged to reduce the workflow runtime when a cache-update isn't required. Seems like only a small adjustment would be needed?
Not sure if this issue could be addressed in this PR too: skipping the post-step altogether when there was a cache-hit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if this issue could be addressed in this PR too
I'm not sure that it can either...
Seeing so many are expecting it, maybe we should explore doing it on another PR?..
Seems like only a small adjustment would be needed?
Sorry @rene-leanix it's been a while - what adjustment would that be?..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@roni-frantchi, I was just referring to @dhadka's comments above in this thread: here and here.
README.md
Outdated
@@ -78,6 +83,8 @@ jobs: | |||
run: /generate-primes.sh -d prime-numbers | |||
|
|||
- name: Use Prime Numbers | |||
env: | |||
CACHE_SKIP_SAVE: true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be removed now that it's set on the cache-primes
step.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Co-authored-by: David Hadka <[email protected]>
@dhadka This is what I wanted #498 for, didn't see this PR though. Right now cache will never be saved if the job fails. Can we add something like: env:
CACHE_FORCE_SAVE: true I think it would be something like this: post-if: success() && env.CACHE_SKIP_SAVE != 'true' || failed() && env.CACHE_FORCE_SAVE == 'true' Thanks. |
Perhaps @ollydev's request could be merged into this one with a single flag?
|
failed() && env.CACHE_FORCE_SAVE == 'true': how do we know the content to cache was generated successfully? I think we need some mechanism to indicate that generating the cache content was successful, and then cache it immediately, why wait until the end of the workflow? |
Other two instances aren't updated. blocked by actions/cache#489 (or 571)
Other two instances aren't updated. blocked by actions/cache#489 (or 571)
@roni-frantchi, is it possible for you to update the branch? It'd remove the deprecation warnings around save-state, set-output and Node.js 12 for those using the fork. |
Hey all, 👋🏽 We have created a discussion with a proposal that will solve this problem, do let us know your feedback, thanks 😊 |
Hey everyone 👋🏽 Two new actions actions/cache/restore and actions/cache/save are now available with tag Do try them and let us know your feedback. We hope these new actions will take care of your use cases. 🙇🏽 Closing this PR now as we believe the new actions will take care of the same, feel free to reopen it or create an issue if need be. |
This version now natively supports read-only caches (`cache/restore@v3`) so we no longer need to pin the version to a commit in actions/cache#489 which is an unmerged PR. The update is mostly mechanical: - Steps with `CACHE_SKIP_SAVE` not set can use the plain `cache@v3` action. - Steps with `CACHE_SKIP_SAVE` set to a constant `true` are changed to `cache/restore@v3`. - Steps with saving disabled when running on a pull request are changed to a pair of `cache/restore@v3` and `cache/save@v3`. This setup is used for the large (100s of MB) ccache and Toolchain caches. As caches saved in pull requests can only be utilized from within the same PR, uploading these would only waste time and our storage quote. Therefore, we skip the `save` steps if running on a PR. Co-authored-by: Cameron Youell <[email protected]>
This version now natively supports read-only caches (`cache/restore@v3`) so we no longer need to pin the version to a commit in actions/cache#489 which is an unmerged PR. The update is mostly mechanical: - Steps with `CACHE_SKIP_SAVE` not set can use the plain `cache@v3` action. - Steps with `CACHE_SKIP_SAVE` set to a constant `true` are changed to `cache/restore@v3`. - Steps with saving disabled when running on a pull request are changed to a pair of `cache/restore@v3` and `cache/save@v3`. This setup is used for the large (100s of MB) ccache and Toolchain caches. As caches saved in pull requests can only be utilized from within the same PR, uploading these would only waste time and our storage quote. Therefore, we skip the `save` steps if running on a PR. Co-authored-by: Cameron Youell <[email protected]>
This version now natively supports read-only caches (`cache/restore@v3`) so we no longer need to pin the version to a commit in actions/cache#489 which is an unmerged PR. The update is mostly mechanical: - Steps with `CACHE_SKIP_SAVE` not set can use the plain `cache@v3` action. - Steps with `CACHE_SKIP_SAVE` set to a constant `true` are changed to `cache/restore@v3`. - Steps with saving disabled when running on a pull request are changed to a pair of `cache/restore@v3` and `cache/save@v3`. This setup is used for the large (100s of MB) ccache and Toolchain caches. As caches saved in pull requests can only be utilized from within the same PR, uploading these would only waste time and our storage quote. Therefore, we skip the `save` steps if running on a PR. Co-authored-by: Cameron Youell <[email protected]>
This is similar to #474 - but improves on that by skipping the
post
(save) step entirely, saving tens of seconds on each run.Fixes #350
Fixes #334