-
Notifications
You must be signed in to change notification settings - Fork 214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cache GMT remote data as artifacts on Github #530
Conversation
Use GMT on macOS to download the GMT remote files from the GMT data server, and then upload them as artifacts on Github.
.github/workflows/ci_tests.yaml
Outdated
github_token: ${{secrets.GITHUB_TOKEN}} | ||
workflow: cache_data.yaml | ||
name: gmt-cache | ||
path: ~/.gmt |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the path correct? Downloading artifacts is fast, but it may not be placed in the correct directories. GMT still tries to download all files during testing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, it looks like it's unzipped to the right location (e.g. ~/.gmt/server/earth/earth_relief/earth_relief_01d_g.grd
), and that's what wget
was doing before?? Maybe we should use $HOME
or something?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you can add a step listing all files in the ~/.gmt directory after downloading the artifacts.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, $HOME doesn't work either. Then I'm not sure where the action is actually unzipping the files to!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've some tests to do, and will make changes to this branch.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, go for it 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see that a ~
folder is created 🤦. How about trying ./.gmt
, since the action uses ./
as the default, see https://github.com/dawidd6/action-download-artifact/blob/2f29da2bb95f5580767170955fca0cf48714d48b/action.yml#L31. Also, we should use ls -alh
to see hidden dot files.
Edit: What am I saying, I should just use .gmt
!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I saw the .gmt
directory in the root directory. At least we can download the artifact, and then move it to ~/
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that's one way of doing it.
So we don't need to set a Github token
Co-Authored-By: Dongdong Tian <[email protected]>
Description of proposed changes
Use GMT on macOS to download the GMT remote files from the GMT data server, and then upload them as artifacts on Github.
Separates the 'cache' task from our Tests CI. Meaning we can update our cache independently of our Continuous Integration tests, allowing the cache to be updated every Sunday for example without needing to manually trigger it with a PR.
References:
Fixes #523, Closes #527
Reminders
make format
andmake check
to make sure the code follows the style guide.doc/api/index.rst
.