Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better error running basin_stats with a data_id not in catalog metadata #148

Closed
jsta opened this issue Nov 3, 2022 · 4 comments
Closed

Comments

@jsta
Copy link
Member

jsta commented Nov 3, 2022

Consider this scenario:

  • a data source like GlobalSurfaceWater Dataset comes out with a new version
  • the catalog watcher removes the old version from the catalog (it still exists on GEE)
  • if you run a query against the old version, you will get a cryptic error (preceded way up the stack by a warning that the id is not in the catalog)

This is because the call to Dataset emits a warning (not an error) and passes on an empty output to basin_stats.compute because the id is not in the catalog (I think this is fine). compute attempts to proceed as normal with the "empty" request (I think this is not fine). We should probably catch these in compute and throw an error.

@jonschwenk
Copy link
Collaborator

Should be easy enough to implement.

For this particular case (and maybe others like it), I think the solution would be to put deprecated assets in rabpro's user_added_catalog. This would avoid errors but this is probably a case where you want it to fail (if you weren't aware of the new version) so maybe that's a bad idea.

@jsta
Copy link
Member Author

jsta commented Dec 5, 2022

Should we enable users to override any checks against the catalog and just run the query? For reasons that are unclear to me, DAYMET does not appear in the scraped json but is clearly available in the GEE catalog.

@jonschwenk
Copy link
Collaborator

Yes, I think so. I was struggling with this for VotE forcings/characteristics fetching as well.

@jsta
Copy link
Member Author

jsta commented Dec 6, 2022

The two dataset attributes that you get "for free" if your entry is in the catalog is type and resolution. I've made it so you can skip dataset validation (with validate_dataset_list=False) and still execute your query by manually specifying these values.

@jsta jsta closed this as completed in 9ef0e26 Dec 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants