Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track pygmt/tests/data using dvc? #1775

Open
maxrjones opened this issue Mar 1, 2022 · 2 comments
Open

Track pygmt/tests/data using dvc? #1775

maxrjones opened this issue Mar 1, 2022 · 2 comments
Labels
discussions Need more discussion before taking further actions longterm Long standing issues that need to be resolved

Comments

@maxrjones
Copy link
Member

The grdtrack tests modified in #1762 require a new .csv file for input. Rather than adding more files to the GMT cache or generating a .csv file within those tests, I am wondering if we can start tracking the folder pygmt/tests/data using dvc? In this case, we could easily add a new file track.txt that contains the necessary points without including it in the git history. This path forward may require some modifications to the dvc-diff workflow and adding these files to the release assets (xref #1317).

@maxrjones maxrjones added the question Further information is requested label Mar 1, 2022
@weiji14
Copy link
Member

weiji14 commented Mar 1, 2022

I had a similar idea at #1695 (comment) with the RidgeTest.shp file to store it using dvc in pygmt/tests/data. In that case, some of the files were binary and not plain-text (shp, shx, dbf), but considering that RidgeTest.shp could be used by GMT/GMT.jl, I think it made sense to have it in https://github.com/GenericMappingTools/gmtserver-admin cache.

For this track.txt file, since it is plain-text, I think we can store it in git history as long as it isn't too long (maybe 10-20 lines). But I think it's worth discussing whether we should look into storing certain binary files using dvc in pygmt/tests/data such as GeoTIFFs in the future, which would require modifications to the dvc workflow as you said, or just have it always on the GMT-wide cache.

@maxrjones
Copy link
Member Author

Sounds good, thanks for the input. I will track that file with git and will leave this issue open for now.

@weiji14 weiji14 added the longterm Long standing issues that need to be resolved label Mar 6, 2022
@seisman seisman added discussions Need more discussion before taking further actions and removed question Further information is requested labels Dec 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussions Need more discussion before taking further actions longterm Long standing issues that need to be resolved
Projects
None yet
Development

No branches or pull requests

3 participants