Broken demo: data download and mean clim #798
Comments
Note for Demo 0Error reproduceimport os
files_md5 = "data_files.txt"
samples = open(files_md5).readlines()
print(samples)
n0 = 1
for sample in samples[n0:]:
print(sample)
good_md5, name = sample.split()ResultError is looking like it was caused by having an empty line at the end of Possible solution
|
|
Demo 0 issue is now resolved by the above solution 1. In addition, in the web server |
|
Demo 1 issue investigation note:
Possible solution:
@gleckler1 Could you please share your insight which of above makes more sense to you? Edit to add: Rather taking either of above option, I updated to download demo_data/obs4MIPs, that the current |
|
Note for continuing repair:
Possible solution:
Edited to add: Resolved by above option 2. |
|
@lee1043 @gleckler1 Regarding the mean climate notebook obs_info_dictionary.json issue, I recall that we updated the obs dictionary to use the obs4mips datasets, but we didn't have a sample data hash file on the pcmdi server that included all the obs4mips data. Does that exist now? |
|
@acordonez yes, I've updated data in the pcmdi server so if you run the current mean clim notebook it should download obs4mips data. |
|
@lee1043 Thanks it looks like the new sample data is working for me. I caught one dataset that uses the "latest" path while the rest use "v20210727" (demo_data/obs4MIPs_PCMDI_daily/NASA-JPL/GPCP-1-3/day/pr/gn/latest/pr_day_GPCP-1-3_PCMDI_gn_19961002-20170101.nc). Would it be possible for them to all use the same convention? Or does daily precip have to be "latest"? |
|
@acordonez thanks for checking it is working and nice catch for about the "latest". I think the original intention was to use the "latest" symbolic linked directory but somehow they were not existing in the pcmdi web server for monthly datasets, which resulted downloading just dummy files. I think it is okay for now as long as the demo is working, but it should be made as consistent. I think that question might be better answered by @gleckler1 |
|
@lee1043 thanks for that reply. I have updated the parameter file templates and notebooks in branch "798_ao_update_nb". Not sure if I should open a PR for this branch, or do you want to merge it into your "787_ljw_pre-commit-hook-compliant" branch? |
|
@acordonez thanks for revising. Either is okay, but merging to master may help closing this issue quickly. |
Demo 0 and Demo 1 are broken regardless of the recent PR for the workflow improvement. In the demos they show following error messages. I suspect an altered directory structure under demo_data may have caused these errors, which need to be checked.
Demo 0, result of cell [3]
Demo 1, result of cell [3]
Above, file looking for in the demo_data directory:
PCMDIobs2_clims/rlut/CERES-EBAF-4-0/v20210804/rlut_mon_CERES-EBAF-4-0_PCMDI_gn.200301-201812.AC.v20210804.ncBut actual file existing in the demo_data directory:
PCMDIobs2_clims/atmos/rlut/CERES-EBAF-4-1/rlut_mon_CERES-EBAF-4-1_BE_gn_200301-201812.v20200421.AC.nc@gleckler1 @acordonez please let me know if you have any thought on this.
The text was updated successfully, but these errors were encountered: