Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop] Streamline SRW App's interface to MET/METplus #1005

Merged
Merged
Show file tree
Hide file tree
Changes from 122 commits
Commits
Show all changes
125 commits
Select commit Hold shift + click to select a range
c54bd0a
Absorb {ACCUM_HH}h in file and directory names for ASNOW into the var…
gsketefian Nov 7, 2023
b766d2c
Partial change from SFC and UPA names to ADPSFC and ADPUPA.
gsketefian Nov 8, 2023
da70c2e
Further modifications for change from SFC and UPA names to ADPSFC and…
gsketefian Nov 8, 2023
aad2eac
Further modifications for renaming from SFC and UPA to ADPSFC and ADP…
gsketefian Nov 8, 2023
a718921
In METplus conf files, change comparison operators in thresholds from…
gsketefian Nov 8, 2023
50dd577
Changes to METplus conf files: (1) FCST_VAR<n>_... and OBS_VAR<n>_...…
gsketefian Nov 8, 2023
cd7b49f
Bug fixes after first round of testing.
gsketefian Nov 8, 2023
5d16582
More bug fixes after first round of testing.
gsketefian Nov 8, 2023
fa51d0a
Use a workflow variable to set the log level in the METplus log files.
gsketefian Nov 9, 2023
e6fe549
Clean up the way the METplus tool names are calculated; pass METplus …
gsketefian Nov 9, 2023
9897066
Change workflow scripts and METplus conf files so that all the vx tas…
gsketefian Nov 11, 2023
cb06fa1
The METplus variables [OBS|FCST]_NATIVE_DATA_TYPE are deprecated and …
gsketefian Nov 11, 2023
76cc60b
Combine METplus config files for APCP01h and APCPgt01h.
gsketefian Nov 13, 2023
fef8d25
For GenEnsProd and EnsembleStat METplus tools, combine conf file temp…
gsketefian Nov 13, 2023
bb570a4
Add changes to ex-script for deterministic GridStat and PoinStat task…
gsketefian Nov 13, 2023
b781e3e
For GridStat_ensmean tasks, combine conf file templates for APCP01h a…
gsketefian Nov 14, 2023
c0ba7fe
Remove commented out lines.
gsketefian Nov 14, 2023
adcd38c
Remove extra whitespace.
gsketefian Nov 14, 2023
8ed996f
Collapse all GridStat_ensprob_APCP*h.conf METplus conf templates into…
gsketefian Nov 14, 2023
6beec34
Remove all use of variable "field_is_APCPgt01h" since the METplus con…
gsketefian Nov 14, 2023
86a4bd6
Use separate cases for specifying thresholds for ASNOW06h and ASNOW24h.
gsketefian Nov 14, 2023
e39ae1c
Make ex-scripts for verification more similar.
gsketefian Nov 14, 2023
8af9285
Minor fix for code appearance consistency.
gsketefian Nov 15, 2023
b57667a
Remove commented out lines in METplus conf files involving [FCST|OBS]…
gsketefian Nov 15, 2023
9585845
Bug fix (OBSVAR7 --> OBS_VAR7); other minor changes to make it easier…
gsketefian Nov 22, 2023
90dcda3
Changes to make it easier to compare GenEnsProd conf files with those…
gsketefian Nov 22, 2023
d4c8502
Changes to make it easier to compare EnsembleStat conf files with tho…
gsketefian Nov 22, 2023
20c1195
Changes to make it easier to compare GridStat_ensprob conf files with…
gsketefian Nov 22, 2023
896e3bd
More changes to make it easier to compare GenEnsProd and EnsembleStat…
gsketefian Nov 22, 2023
19152ee
In METplus config template files that use "BOTH_VAR<n>_...", split th…
gsketefian Nov 22, 2023
8b6a6fd
Add comments and remove extra whitespace.
gsketefian Nov 22, 2023
9c2c8db
Clean up comments for forecast and obs time windows.
gsketefian Nov 23, 2023
3be4460
Make comments for logging consistent with other conf files.
gsketefian Nov 23, 2023
bced894
Clean up comments, put each option on separate line, remove unneeded …
gsketefian Nov 23, 2023
4c7f88b
Make the thresholds for CAPE in GenEnsProd_ADPUPA.conf consistent wit…
gsketefian Nov 23, 2023
80741d2
Undo previous commit since we want to do it later as a non-result cha…
gsketefian Nov 23, 2023
e919360
Move THRESH variable to before OPTIONS and after LEVELS.
gsketefian Nov 23, 2023
13b7095
Clean up comments.
gsketefian Nov 23, 2023
e9a5368
Fix up comments.
gsketefian Nov 23, 2023
0818802
Set verbosity level of PcpCombine tasks using workflow variable; remo…
gsketefian Nov 23, 2023
28aafdb
Fix up comments.
gsketefian Nov 23, 2023
9728bdb
Clean up comments.
gsketefian Nov 23, 2023
fd58d4e
Remove use of BOTH_ variables in deterministic conf files; modify com…
gsketefian Nov 25, 2023
3e1b8f9
Clean up comments; move around variable settings for consistency with…
gsketefian Nov 25, 2023
d35a7e1
Fixes in comments and spacing.
gsketefian Nov 25, 2023
7305347
Remove all commented-out lines involving variables named "..._FILE_TY…
gsketefian Nov 25, 2023
fda85e8
Changes to comments.
gsketefian Nov 25, 2023
f356a6c
Remove variable ACCUM_HH in METplus conf files (and replace with jinj…
gsketefian Nov 25, 2023
0d535b5
Minor comment fixes.
gsketefian Nov 25, 2023
efcc635
Minor comment fixes.
gsketefian Nov 25, 2023
7e3b9ac
Minor comment fixes.
gsketefian Nov 25, 2023
780671c
Remove extraneous field suffix in name of PcpCombine METplus conf files.
gsketefian Nov 27, 2023
d2a80ef
Move comment for an option to end of line.
gsketefian Nov 27, 2023
95d492f
Move comments for options to end of line.
gsketefian Nov 27, 2023
7fe7af9
Remove internal variable FIELDNAME_IN_MET_FILEDIR_NAMES in EnsembleSt…
gsketefian Nov 27, 2023
2750d49
Remove internal variable FIELDNAME_IN_MET_FILEDIR_NAMES in EnsembleSt…
gsketefian Nov 27, 2023
e134583
Change spacing to make it more convenient to compare to output from o…
gsketefian Nov 27, 2023
314f773
Spacing fix.
gsketefian Nov 28, 2023
a2883de
Spacing changes.
gsketefian Nov 28, 2023
eea96dd
In the METplus conf files for GridStat, GridStat_ensmean (which is no…
gsketefian Nov 28, 2023
ce2f098
Remove comented-out lines.
gsketefian Nov 28, 2023
dfd21e3
For REFC and RETOP, modify the METplus conf files for GridStat, GridS…
gsketefian Nov 28, 2023
fb7bc8c
Include files inadvertently left out of previous commit.
gsketefian Nov 28, 2023
8e012d6
Merge branch 'feature/vx_upgrades' of https://github.com/gsketefian/u…
gsketefian Nov 29, 2023
a0e3d99
Add the accumulation period (in hours) as a suffix to the names of AP…
gsketefian Jan 4, 2024
52cf89b
Bug fix. Changes that should have been made as part of previous comm…
gsketefian Jan 4, 2024
26aaf87
For simplicity, remove ".0" from REFC and RETOP thresholds.
gsketefian Jan 4, 2024
2f5fcde
Add the accumulation hour as a suffix to the ASNOW forecast variable …
gsketefian Jan 5, 2024
935cf5c
Merge remote-tracking branch 'origin/feature/vx_upgrades' into featur…
gsketefian Jan 5, 2024
1761430
Merge branch 'develop_upstream' into feature/vx_upgrades
gsketefian Jan 9, 2024
cd94c85
Fix typo.
gsketefian Jan 9, 2024
b2c78eb
Merge branch 'develop_upstream' into feature/vx_upgrades
gsketefian Jan 11, 2024
2743515
Bug fixes for ensemble vx of ASNOW.
gsketefian Jan 12, 2024
b0d53be
Edit comments and rearrange some code in METplus conf files.
gsketefian Jan 16, 2024
18c26ff
Merge branch 'develop' into feature/metplus_conf_templates
gsketefian Jan 16, 2024
a0e4d42
Add new files needed for jinja-fication of METplus conf files.
gsketefian Jan 18, 2024
32cb427
Changes to ex-script for deterministic GridStat and PointStat tasks t…
gsketefian Jan 18, 2024
706ad2d
Changes to make deterministic GridStat and PointStat tasks work using…
gsketefian Jan 18, 2024
202ba9a
Add new METplus conf file templates that are generalized over the fie…
gsketefian Jan 19, 2024
0cc2bac
Modify ex-scripts for vx tasks so they work with new METplus config f…
gsketefian Jan 19, 2024
08a463e
Modify yaml config file for workflow vx tasks to that the rocoto xml …
gsketefian Jan 19, 2024
63fa55a
Revert back to the develop branch certain changes that were inadverta…
gsketefian Jan 19, 2024
a854f6a
dd new METplus PcpCombine conf file template that is generalized over…
gsketefian Jan 19, 2024
62eb1b1
Add file left out of last commit.
gsketefian Jan 19, 2024
5108253
Remove unneeded files.
gsketefian Jan 20, 2024
e4ec134
Modify METplus config file templates for ensemble vx tasks to enable …
gsketefian Jan 20, 2024
52f4921
Modify yaml file for ensemble verification to enable ASNOW verification.
gsketefian Jan 20, 2024
2b80c16
Modify yaml config file for ensemble workflow vx tasks to add variabl…
gsketefian Jan 20, 2024
dcdd67b
Minor spacing changes.
gsketefian Jan 20, 2024
90895b0
Spacing fix.
gsketefian Jan 20, 2024
52678b4
Move python scripts borrowed from uw_tools into separate subdirectory…
gsketefian Jan 22, 2024
bfcce00
Remove inadvertantly added log file.
gsketefian Jan 22, 2024
1b5f25a
Remove comment in GridStat_ensprob.conf section containing METplus va…
gsketefian Jan 23, 2024
e38db7b
Fixes to comments in PcpCombine.conf.
gsketefian Jan 23, 2024
3b8fbbb
Remove unnecessary code involving the outdated variable FIELD_THRESHO…
gsketefian Jan 24, 2024
5f69610
For writing the contents of the $settings variable to tmpfile, use ba…
gsketefian Jan 24, 2024
16b4d7b
Merge branch 'develop' into feature/metplus_conf_templates
gsketefian Feb 16, 2024
fa196b5
Track file inadvertantly left out of version control.
gsketefian Feb 28, 2024
eb58117
Remove deprecated config settings in the EnsembleStat config template…
gsketefian Mar 19, 2024
1c65d35
Remove unnecessary variable setting in pcp_combine ex-script.
gsketefian Mar 19, 2024
b25d148
Add code to ex-script for gridstat and pointstat to quickly switch be…
gsketefian Mar 20, 2024
2ca4c51
Merge branch 'develop' into feature/metplus_conf_templates
gsketefian Mar 20, 2024
326b714
Change scripts for vx to always use new uwtools.
gsketefian Mar 20, 2024
7e0974d
Clean up unnecessary code from ex-scripts and remove directory contai…
gsketefian Mar 20, 2024
f745894
Remove debugging print.
gsketefian Mar 21, 2024
7074362
Use distinct files for the forecast-obs separated vx configuration fi…
gsketefian Apr 1, 2024
0bd54b9
Merge branch 'develop' into feature/metplus_conf_templates
gsketefian Apr 20, 2024
59f6241
Merge branch 'develop' into feature/metplus_conf_templates
gsketefian Apr 24, 2024
3272b4c
Remove unused macro in jinja2 macros file.
gsketefian Apr 26, 2024
c078319
Move the script separate_fcst_obs_info.py from the parm directory to …
gsketefian Apr 26, 2024
e8ded6e
Add documentation to the configuration files for deterministic and en…
gsketefian Apr 26, 2024
f7e6f97
Fix comments in j-job for PB2NC vx task.
gsketefian Apr 29, 2024
552376a
Overhaul and rename script separate_fcst_obs_info.py for parsing vx c…
gsketefian Apr 29, 2024
c6dd630
Modify vx ex-scripts so they only read in the vx configuration text f…
gsketefian Apr 29, 2024
9374703
Put in option to not alphabetically reorder dictionaries (and lists) …
gsketefian Apr 29, 2024
b790f57
Update comments in J-job and ex-script for check_post_output_... tasks.
gsketefian Apr 29, 2024
f0dcde7
Bug fixes and renaming entities in workflow config files for vx.
gsketefian Apr 29, 2024
f6cd1db
Add vx tasks for parsing vx configuration files.
gsketefian Apr 29, 2024
ca76455
Spacing changes.
gsketefian Apr 29, 2024
502d4ff
Change code snippet for parsing "_cpld" items to that suggested by Mi…
gsketefian Apr 30, 2024
d4c5728
Bug fix.
gsketefian Apr 30, 2024
8263603
Fix typo.
gsketefian Apr 30, 2024
b6bce46
Merge branch 'develop' into feature/metplus_conf_templates
gsketefian Apr 30, 2024
3fad014
Add j-job and ex-script for parse_vx_config_[det|ens] tasks.
gsketefian Apr 30, 2024
a7f84c2
Shorten comments to address Mike K's PR comments.
gsketefian May 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions jobs/JREGIONAL_CHECK_POST_OUTPUT
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,11 @@ print_info_msg "
Entering script: \"${scrfunc_fn}\"
In directory: \"${scrfunc_dir}\"

This is the J-job script for the task that checks that all the post-
processed output files in fact exist and are at least a certain age.
These files may have been generated by UPP as part of the current SRW
App workflow, or they may be user-staged.
This is the J-job for the task that checks that no more than
NUM_MISSING_FCST_FILES_MAX of each forecast's (ensemble member's) post-
processed output files are missing. Note that such files may have been
generated by UPP as part of the current SRW App workflow, or they may be
user-staged.
========================================================================"
#
#-----------------------------------------------------------------------
Expand All @@ -70,9 +71,9 @@ Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
#
#-----------------------------------------------------------------------
#
# Create a flag file to make rocoto aware that the make_grid task has
# successfully completed (so that other tasks that depend on it can be
# launched).
# Create a flag file to make rocoto aware that the check_post_output task
# has successfully completed (so that other tasks that depend on it can
# be launched).
#
#-----------------------------------------------------------------------
#
Expand Down
4 changes: 2 additions & 2 deletions jobs/JREGIONAL_RUN_MET_PB2NC_OBS
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,8 @@ print_info_msg "
Entering script: \"${scrfunc_fn}\"
In directory: \"${scrfunc_dir}\"

This is the J-job script for the task that runs METplus for point-stat
by initialization time for all forecast hours.
This is the J-job script for the task that runs METplus for pb2nc on
NDAS observations.
========================================================================"
#
#-----------------------------------------------------------------------
Expand Down
Loading
Loading