[develop] Streamline SRW App's interface to MET/METplus (ufs-communit…

…y#1005)

This PR streamlines the SRW App's interface to the MET/METplus verification tool and implements some bug fixes. Details:
* Replace the field-specific METplus configuration jinja2 templates associated with each METplus tool (these templates are hard-coded for each field) with a single template that contains jinja2 code to any valid field to be verified.
* Add yaml configuration files for verification that specify the fields to verify (including field levels and thresholds). This is in order to consolidate the field/level/threshold information in one place instead of having it spread out and repeated in several hard-coded configuration files.
* Add a python script (decouple_fcst_obs_vx_config.py) to parse these two vx configuration files and create a dictionary of the field/level/threshold information that can then be passed to the unified workflow templating tool.
* Modify the ex-scripts for the verification tasks (exregional_run_met_....sh) to allow the use of the new jinja2 METplus config templates. This includes adding code to call the new script decouple_fcst_obs_vx_config.py and then passing its output to the unified workflow templating tool to generate METplus configuration files from the (new) jinja2 templates.
* Add new environment variables to the rocoto workflow configuration files (verify_[pre|det|ens].yaml) that are needed for using the new jinja2 METplus config templates.
* Bug fixes

jobs/JREGIONAL_CHECK_POST_OUTPUT

@@ -51,10 +51,11 @@ print_info_msg "
 Entering script: \"${scrfunc_fn}\"
 In directory: \"${scrfunc_dir}\"
 
-This is the J-job script for the task that checks that all the post-
-processed output files in fact exist and are at least a certain age.
-These files may have been generated by UPP as part of the current SRW
-App workflow, or they may be user-staged. 
+This is the J-job for the task that checks that no more than 
+NUM_MISSING_FCST_FILES_MAX of each forecast's (ensemble member's) post-
+processed output files are missing. Note that such files may have been
+generated by UPP as part of the current SRW App workflow, or they may be
+user-staged.
 ========================================================================"
 #
 #-----------------------------------------------------------------------
@@ -70,9 +71,9 @@ Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
 #
 #-----------------------------------------------------------------------
 #
-# Create a flag file to make rocoto aware that the make_grid task has 
-# successfully completed (so that other tasks that depend on it can be
-# launched). 
+# Create a flag file to make rocoto aware that the check_post_output task
+# has successfully completed (so that other tasks that depend on it can
+# be launched). 
 #
 #-----------------------------------------------------------------------
 #

jobs/JREGIONAL_PARSE_VX_CONFIG

@@ -0,0 +1,97 @@
+#!/usr/bin/env bash
+
+#
+#-----------------------------------------------------------------------
+#
+#
+#
+#-----------------------------------------------------------------------
+#
+
+#
+#-----------------------------------------------------------------------
+#
+# Source the variable definitions file and the bash utility functions.
+#
+#-----------------------------------------------------------------------
+#
+. $USHdir/source_util_funcs.sh
+source_config_for_task "task_parse_vx_config" ${GLOBAL_VAR_DEFNS_FP}
+. $USHdir/job_preamble.sh
+#
+#-----------------------------------------------------------------------
+#
+# Save current shell options (in a global array). Then set new options
+# for this script/function.
+#
+#-----------------------------------------------------------------------
+#
+{ save_shell_opts; . $USHdir/preamble.sh; } > /dev/null 2>&1
+#
+#-----------------------------------------------------------------------
+#
+# Get the full path to the file in which this script/function is located 
+# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in
+# which the file is located (scrfunc_dir).
+#
+#-----------------------------------------------------------------------
+#
+scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" )
+scrfunc_fn=$( basename "${scrfunc_fp}" )
+scrfunc_dir=$( dirname "${scrfunc_fp}" )
+#
+#-----------------------------------------------------------------------
+#
+# Print message indicating entry into script.
+#
+#-----------------------------------------------------------------------
+#
+print_info_msg "
+========================================================================
+Entering script: \"${scrfunc_fn}\"
+In directory: \"${scrfunc_dir}\"
+
+This is the J-job script for the task that reads in the \"coupled\" yaml
+verification (vx) configuration file (python dictionary) and generates
+from it two \"decoupled\" vx configuration dictionaries, one for forecasts
+and another for observations. The task then writes these two decoupled
+dictionaries to a new configuration file in the experiment directory
+that can be read by downstream vx tasks.
+
+Note:
+The \"coupled\" vx configuration file contains items (dictionary keys and
+values representing field names, levels, and thresholds) that consist of
+both the forecast and the observation value for that item separated by a
+delimiter string. Thus, they first need to be separated (decoupled) into
+a value for forecasts and one for the observations before they can be
+further processed.
+========================================================================"
+#
+#-----------------------------------------------------------------------
+#
+# Call the ex-script for this J-job and pass to it the necessary varia-
+# bles. 
+#
+#-----------------------------------------------------------------------
+#
+$SCRIPTSdir/exregional_parse_vx_config.sh || \
+print_err_msg_exit "\
+Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
+#
+#-----------------------------------------------------------------------
+#
+# Run job postamble.
+#
+#-----------------------------------------------------------------------
+#
+job_postamble
+#
+#-----------------------------------------------------------------------
+#
+# Restore the shell options saved at the beginning of this script/func-
+# tion.
+#
+#-----------------------------------------------------------------------
+#
+{ restore_shell_opts; } > /dev/null 2>&1
+

jobs/JREGIONAL_RUN_MET_PB2NC_OBS

@@ -51,8 +51,8 @@ print_info_msg "
 Entering script: \"${scrfunc_fn}\"
 In directory: \"${scrfunc_dir}\"
 
-This is the J-job script for the task that runs METplus for point-stat
-by initialization time for all forecast hours.
+This is the J-job script for the task that runs METplus for pb2nc on
+NDAS observations.
 ========================================================================"
 #
 #-----------------------------------------------------------------------