Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to WW3 #5

Closed
JessicaMeixner-NOAA opened this issue Nov 1, 2019 · 1 comment
Closed

Updates to WW3 #5

JessicaMeixner-NOAA opened this issue Nov 1, 2019 · 1 comment

Comments

@JessicaMeixner-NOAA
Copy link
Collaborator

  • Updated WW3 which includes updates for:
    -- hera support
    -- Adding capability for interpolation from regular grid to curvilinear grids
    -- Switch WRST for storing wind in the restart and using it for the first time step in the NUOPC cap.
  • NEMSCompsetRun tests with WW3 run on hera
  • updates so that wave coupling is tested in the rt.sh tests
  • WW3 output added to baselines for checking results

Currently these updates can be found in: https://github.com/JessicaMeixner-NOAA/ufs-weather-model/tree/ww3update

@junwang-noaa
Copy link
Collaborator

This is committed to the repo. Close the issue.

climbfuji referenced this issue in climbfuji/ufs-weather-model Jan 22, 2020
* Removing WW3 submodule to point to new branch

* Changing WW3 repo commit to latest in branch GEFS_v12

* Changing WW3 repo commit to latest hash in branch GEFS_v12

* Added GFSv16 operational grids to c768 compset. Changed resources required by waves for this compset to fit in 1800s runtime window.

* Adding dell for allowing running c768 compsets

* Adding parameter C768_THRD for expanding threads due to use of higher res GFSv16-wave grids in compset. Updating logs, adjusting c768 resources for Hera.

* Changing resource requirements of c768-ww3 compset to allow faster access to batch queues. Change aligns resource requirements to GFSv16 FV3 and WW3 configs.

* Adding logs after successfull runs with new resource settings for c768

* Setting up for updating WW3 submodule

*  Pointing submodule WW3 to latest develop hash.

* Updating to latest FV3 from dusan/gfsv16_netcdf_post_ficein_cpl

* Updating FMS and stochastic_physics module to latest hash

* Updating FV3 to match the develop branch. Previous update to dusan fv3atm repo caused conflict.

* Extending time limit for execution of c768 on Dell

* Updating to latest WW3 develop

* Adding new hera logs for ww3 compsets

* Adding new Cray logs for ww3 compsets

* Updating to latest WW3 develop: bugfixes to the wave components for running under the ufs-weather-model.

* Adding new WW3 develop hash. Changing to C768_THRD=1 to ensure b4b reproducibility of ww3 tests.
climbfuji referenced this issue in climbfuji/ufs-weather-model Aug 7, 2020
XiaSun-Atmos pushed a commit to XiaSun-Atmos/ufs-weather-model that referenced this issue Aug 8, 2020
, Add 3D reflectivity to restart file and restart reproducibility for regional fv3,3)ufs issue ufs-community#5, Updates to WW3, 4)vlab #69735, update netcdf time units attribute when iau_offset
XiaSun-Atmos pushed a commit to XiaSun-Atmos/ufs-weather-model that referenced this issue Aug 8, 2020
Bugfixes in inline post, netcdf time stamp and gfs physics for coupled system. GFS physics update to reduce the cold bias in lower atmosphere layer. VLAB tickets: #69814 and #69735. fv3atm issue: ufs-community#4, ufs-community#5, ufs-community#6 and ufs-community#7.
pjpegion added a commit to mlee03/ufs-weather-model that referenced this issue Feb 26, 2021
LarissaReames-NOAA pushed a commit to LarissaReames-NOAA/ufs-weather-model that referenced this issue Oct 22, 2021
…gmtb_develop_20191016

Update master from gmtb/develop 2019/10/16
LarissaReames-NOAA pushed a commit to LarissaReames-NOAA/ufs-weather-model that referenced this issue Oct 22, 2021
- bugfix for GNU compiler
- reduce noise in stdout

Squashed commit of the following:

commit 86c127f5b810b87004aeed3a625f9445d469c27e
Merge: ce66d6a f9bbb4f
Author: Dom Heinzeller <[email protected]>
Date:   Fri Apr 10 11:32:21 2020 -0600

    Merge pull request ufs-community#11 from climbfuji/update_dtc_develop_from_dev_emc_20200409

    Update dtc/develop from dev/emc 2020/04/09

commit f9bbb4f12e032378050cef4b7e5ac4b65a16bb70
Author: Dom Heinzeller <[email protected]>
Date:   Thu Apr 9 20:14:24 2020 -0600

    Bugfix for GNU compiler in model/fv_regional_bc.F90, reduce verbosity in driver/fvGFS/atmosphere.F90

commit ecca9978802ec9cc39dc114fd7363386f7ad9422
Merge: ce66d6a 80ce8ce
Author: Dom Heinzeller <[email protected]>
Date:   Thu Apr 9 09:38:48 2020 -0600

    Merge branch 'dev/emc' of https://github.com/NOAA-EMC/GFDL_atmos_cubed_sphere into HEAD

commit ce66d6a56e89b6330a2bef763755c7154047d829
Merge: 4279bf7 9bacf55
Author: Dom Heinzeller <[email protected]>
Date:   Fri Mar 13 13:00:41 2020 -0600

    Merge pull request ufs-community#10 from climbfuji/update_from_dev_emc_20200312_and_other_changes

    Update from dev/emc 2020/03/12

commit 9bacf554ed078dfa8c09ee662e7608442288e2d5
Merge: 4279bf7 371a29a
Author: Dom Heinzeller <[email protected]>
Date:   Thu Mar 12 10:33:44 2020 -0600

    Merge branch 'dev/emc' of https://github.com/NOAA-EMC/GFDL_atmos_cubed_sphere into HEAD

commit 4279bf78eb4ae0bad908eb991e3fb216018b7aea
Merge: 846f5a5 4858d33
Author: Dom Heinzeller <[email protected]>
Date:   Mon Mar 9 12:41:51 2020 -0600

    Merge pull request ufs-community#9 from climbfuji/update_dtc_develop_from_emc

    Update dtc/develop from dev/emc 2020/03/04

commit 4858d33838adcd87c1cd000ab4737dc0e7e56731
Merge: 846f5a5 db3acfb
Author: Dom Heinzeller <[email protected]>
Date:   Fri Feb 28 08:46:55 2020 -0700

    Merge branch 'dev/emc' of https://github.com/NOAA-EMC/GFDL_atmos_cubed_sphere into HEAD

commit 846f5a54b0764db1478c9aca49a74bd38322d9ad
Merge: 3a4dfd8 9a290ee
Author: Dom Heinzeller <[email protected]>
Date:   Mon Feb 3 07:43:08 2020 -0700

    Merge pull request ufs-community#7 from climbfuji/update_dtc_develop_from_dev-emc

    dtc/develop: update from dev/emc 2020/01/27

commit 9a290ee5f3307a9d7b02762fe6c29a1f5ccd7f55
Merge: 3a4dfd8 a56907a
Author: Dom Heinzeller <[email protected]>
Date:   Mon Jan 27 09:28:57 2020 -0700

    Merge branch 'dev/emc' of https://github.com/noaa-emc/GFDL_atmos_cubed_sphere into HEAD

commit 3a4dfd8c6c4ceb8cec06397f25cb229ecd98065b
Merge: 0c9ab9e 68576a6
Author: Dom Heinzeller <[email protected]>
Date:   Tue Dec 3 15:19:24 2019 -0700

    Merge pull request ufs-community#6 from climbfuji/dtc_develop_udpate_from_emc_20191127

    dtc/develop: update from EMC 2019/11/27

commit 68576a61f2a57236931e5ef1b8bd73a4256b0a5f
Merge: 0c9ab9e 452333a
Author: Dom Heinzeller <[email protected]>
Date:   Wed Nov 27 14:13:08 2019 -0700

    Merge branch 'dev/emc' of https://github.com/NOAA-EMC/GFDL_atmos_cubed_sphere into HEAD

commit 0c9ab9e3dfe6cda6fc78315b86a107a369756024
Merge: 9871607 b280b37
Author: Dom Heinzeller <[email protected]>
Date:   Fri Nov 22 16:17:57 2019 -0700

    Merge pull request ufs-community#5 from mzhangw/HAFS_fer_hires

    minimal changes to make nwat =4 compatible with Ferrier-Aligo MP scheme

commit b280b373b421fcb9d84bd1196a40ec56fcf279ff
Author: Man.Zhang <[email protected]>
Date:   Tue Nov 19 14:53:57 2019 -0700

    bug fix

commit 98c58ad3d0444859aae54c31bc2b86ba30b4a9ac
Author: Man.Zhang <[email protected]>
Date:   Tue Nov 19 14:35:10 2019 -0700

    bug fix

commit 8805acd93152bb76d5c4d0900f314631f549fbe2
Author: Man.Zhang <[email protected]>
Date:   Tue Nov 19 14:16:22 2019 -0700

    From Chunxi: The file fv_mapz.F90 also needs to be modified (K_warm)

commit f8a257a922a9047e98c176affec054e2a4d9c6a9
Author: Man.Zhang <[email protected]>
Date:   Thu Nov 14 19:12:39 2019 -0700

    use upper-case CCPP

commit 77f4fbad9311ad406c6486ad0912069de3085f07
Author: Man.Zhang <[email protected]>
Date:   Thu Nov 14 13:40:59 2019 -0700

     implement FA scheme water loading option if nwat =4

commit 987160799b3a1c0f11de9883843f20421eb03fe8
Merge: ed75004 786447c
Author: Dom Heinzeller <[email protected]>
Date:   Tue Nov 5 06:59:34 2019 +0900

    Merge branch 'dev/emc' of https://github.com/NOAA-EMC/GFDL_atmos_cubed_sphere into HEAD

commit ed75004634374b02f8b222dd873190a91126ce4f
Merge: 1d6035a f627bfb
Author: Dom Heinzeller <[email protected]>
Date:   Sat Oct 12 08:45:14 2019 +0900

    Merge pull request ufs-community#4 from climbfuji/update_gmtb_develop_with_vlab_master_20191006

    Update NCAR gmtb/develop with NOAA-EMC dev/emc 2019/10/06

commit f627bfb177dc1f2d0bd10371540c43c51ce8eec4
Author: Dom Heinzeller <[email protected]>
Date:   Mon Oct 7 12:12:24 2019 +0900

    Add missing change in fv_cmp.F90 - now identical with EMC code

commit e868186fd7d64fc2eeb7ec2c59e88c889e62fc93
Author: Dusan Jovic <[email protected]>
Date:   Tue Sep 24 01:57:11 2019 +0000

    change delz from positive value to the original value in the model

commit 672eb0e8d67c008437724681250e7aa8aa81ec31
Author: fanglin.yang <[email protected]>
Date:   Thu Sep 5 19:24:09 2019 +0000

        VLab Issue #68141

        modified:   docs/FV3_citations.bib
        modified:   model/fv_cmp.F90

        1) Add an option (namelist parameter: intqs) to use temperature instead of the liquid frozen
        temperature to calculate saturation mixing ratio in deriving GFDLMP PDF cloud cover.
        2) Add a literature reference.

commit 1d6035ab053a6b5257b32dd9b4a49faa7ee4c78d
Merge: 94ab0e1 fec8205
Author: Dom Heinzeller <[email protected]>
Date:   Mon Sep 23 09:32:55 2019 -0600

    Merge pull request ufs-community#2 from climbfuji/remove_transition_mode

    Remove TRANSITION mode

commit fec8205b9418dea28de4e2c9f05f347f3cf460d2
Author: climbfuji <[email protected]>
Date:   Fri Sep 20 08:52:01 2019 -0600

    Remove TRANSITION mode

commit 94ab0e1317ffa50279dd6269a462999a2bea15e7
Author: Jun.Wang <[email protected]>
Date:   Mon Aug 26 21:03:55 2019 +0000

    add option to output omega

commit b700cae673038f82189a1e6e0cb48a10a58ea726
Author: climbfuji <[email protected]>
Date:   Fri Aug 30 11:58:00 2019 -0600

    model/fv_dynamics.F90: replicating previous change to gmtb/develop branch of FV3, always allocate dtdt_m for IPD version

commit 3df11713e0ba9299d95512476a9ee784048629c9
Author: Laurie Carson <[email protected]>
Date:   Fri Aug 16 10:46:56 2019 -0600

    Add codeowners for NCAR fork
epic-cicd-jenkins pushed a commit that referenced this issue Apr 17, 2023
epic-cicd-jenkins pushed a commit that referenced this issue Apr 17, 2023
 - Adds a CMake build using ExternalProject_Add.
 - Makes modifications to build_all.sh and install_all.sh for back-compatibility with old build system. The old build system will remain in, but is now deprecated, and will be removed in the coming weeks.

To build, have the necessary modules loaded and run:

```
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=<prefix>
make
```

The docs/ directory contains README files with instructions for specific platforms/compilers

 - **Cheyenne**: Build and end-to-end test ("DOT_OR_USCORE" test case) was successful on Cheyenne with intel, both for the cmake build and the old build script (that will soon be deprecated). Path to tests: /glade/scratch/kavulich/UFS_CAM/testing/SRW_PR_27/expt_dirs/
 - **Hera**: Build and end-to-end tests successful (aside from expected failures). Path to tests: /scratch2/BMC/det/kavulich/workdir/SRW_PR_27/expt_dirs
 - **Jet**: Build test was successful.

Resolves #7, partially resolves #5

Co-authored-by: kgerheiser <[email protected]>; Michael Kavulich, Jr <[email protected]>
epic-cicd-jenkins pushed a commit that referenced this issue Apr 17, 2023
* Fix to post flat file.

* Create MET and METplus config files under ush/templates/parm

* Added script to pull and reorg ccpa data. Added a script to run gridstat with METplus. Updated MET and METplus config files.

* Added new jjob for running grid-stat vx. Updated setup.sh to include grid-stat vx. Updated run_gridstatvx script.

* Fixed typo on script name from ksh to sh

* Moved some hard coded items out from the script to the XML

* Updates to get METplus to run with fewer hard-coded paths.

* Updates to add grid-stat task to XML generation.

* Bug fixes for adding grid-stat to XML generation

* Updates to remove hard-coded paths in config files

* Change log dir to put master_metplus log file with other logs under log/, rather than default logs/.

* Updates to generate xml without hard-coded paths for MET

* Add hera gridstat module file

* Add METplus point-stat task for both sfc and upper air

* Small tweaks to remove hard coded paths and add some flexibility

* Updates for adding point-stat into auto-generated xml

* Add in function to set point-stat task to FALSE

* Final tweaks to get it to generate the xml correctly

* Minor updates to run ensure 0,6,12,18

* Tweaks to var list for Point-Stat

* Add METplus settings to config_defaults

* Move quote for end of settings and fix extra comment.

* Fix typos to populate templates correctly

* Updated to include SCRIPTSDIR and other MET specific settings along with updates to FHR syntax

* Update module loads on hera

* Fixed comment for BOTH_VARn_THRESH to avoid syntax issues

* Added files to run grid_stat for a variety of accumulation intervals, including 3, 6, and 24h

* Added module load hpss

* Remove module load informatino from these scripts

* Updated the method of turning on/off vx tasks using jinja template if statement

* Remove commented out lines of code. Fixed typo. Removed gen_wflow.out file.

* Updated pull scripts to have file names dependent on date to pull from HPSS. Updated to export a few more local variables that METplus conf needed in scripts. Updated workflow to use service queue (for now) to for 1h grid_stat and point_stat run and default for 3+h accumulation grid_stat runs)

* moved common_hera.conf to common.conf - no platform specific information included that needs to be handled.

* Remove common_hera.conf

* Add scripts to pull and process MRMS data from NOAA HPSS

* Updates for REFC vx tasks

* updates to obs pull scripts

* Update for adding in reflectivity verification using MRMS analyses and updating name of model output to RRFS rather than HRRR

* Updates to account for CCPA issues on HPSS - day off for 00-05 UTC directories

* Verification mods to feature/add metplus (#1)

* Remove unused/outdated code (#313)

## DESCRIPTION OF CHANGES:
* In setup.sh and generate_FV3LAM_wflow.sh, remove temporary codes that fix bugs in the FV3_GFS_2017_gfdlmp_regional suite definition file because those bugs have been fixed (in the ufs-weather-model repo).
* In setup.sh, remove block of code that is no longer necessary because chgres_cube can now initialize from external model data with either 4 or 9 soil levels, and run with LSMs of either 4 or 9 soil levels.
* Remove modifications to LD_LIBRARY_PATH in exregional_run_fcst.sh.
* For the make_ics and make_lbcs tasks, move the setting of APRUN and other machine-specific actions from the J-job to the ex-script in order to be consistent with the other workflow tasks.
* Fix indentation and edit comments.
* Remove unused file load_fv3gfs_modules.sh.

## TESTS CONDUCTED: 
Ran two WE2E tests on hera, new_ESGgrid and new_GFDLgrid:
* new_ESGgrid uses the FV3_GFS_2017_gfdlmp_regional suite.  The test was successful.
* new_GFDLgrid uses the FV3_GFS_2017_gfdlmp suite.  The test was successful.

## ISSUE (optional): 
This resolves issue #198.

* Add and call a function that checks for use of Thompson microphysics parameterization in the SDF and if so, adjusts certain workflow arrays to contain the names and other associated values of the fixed files needed by this parameterization so that those files are automatically copied and/or linked to. (#319)

## DESCRIPTION OF CHANGES: 
Add and call a function that checks for use of Thompson microphysics parameterization in the suite definition file (SDF).  If not, do nothing.  If so, add to the appropriate workflow arrays the names and other associated values of the fixed files needed by this parameterization so that they are automatically copied and/or linked to instead of being regenerated from scratch in the run_fcst task.

## TESTS CONDUCTED: 
On hera, ran two WE2E tests, one in NCO mode (nco_RRFS_CONUS_25km_HRRRX_RAPX) and the other in community mode (suite_FV3_GSD_v0).  These use suites FV3_GSD_SAR and FV3_GSD_v0, respectively, and both of these call Thompson microphysics.  Both succeeded.

## ISSUE (optional):
This PR resolves issue #297.

* RRFS_v1beta SDF changes after reverting from GSL to GFS GWD suite (#322) (#327)

## DESCRIPTION OF CHANGES:
Removed checks on the RRFS_v1beta SDF implemented for use with the GSL GWD suite (now uses the GFS GWD suite).  No longer copies staged orography files necessary for the GSL GWD suite.

## TESTS CONDUCTED:
Runs to completion on Hera. End-to-end runs DOT_OR_USCORE and suite_FV3_RRFS_v1beta succeeded on Cheyenne.

Co-authored-by: JeffBeck-NOAA <[email protected]>

* Update FV3.input.nml for fhzero = 1.0

* Updated conf files for file name conventions.

* Updated MET scripts and MRMS pull scripts.

* Adjust RRFS_CONUS_... grids (#294)

## DESCRIPTION OF CHANGES: 
* Adjust RRFS_CONUS_25km, RRFS_CONUS_13km, and RRFS_CONUS_3km grid parameters so that:
  * All grids, including their 4-cell-wide halos, lie completely within the HRRRX domain.
  * All grids have dimensions nx and ny that factor "nicely", i.e. they don't have factors greather than 7.
  * The write-component grids corresponding to these three native grids cover as much of the native grids as possible without going outside of the native grid boundaries.  The updated NCL scripts (see below) were used to generate the write-component grid parameters.
* For the RRFS_CONUS_13km grid, reduce the time step (DT_ATMOS) from 180sec to 45sec.  This is necessary to get a successful forecast with the GSD_SAR suite, and thus likely also the RRFS_v1beta suite.
* Modify WE2E testing system as follows:
  * Add new tests with the RRFS_CONUS_25km, RRFS_CONUS_13km, and RRFS_CONUS_3km grids that use the GFS_v15p2 and RRFS_v1beta suites (which are now the ones officially supported in the first release of the short-range weather app) instead of the GFS_v16beta and GSD_SAR suites, respectively.
  * For clarity, rename the test configuration files that use the GFS_v16beta and GSD_SAR suites so they include the suite name.
  * Update list of WE2E tests (baselines_list.txt).
* Update the NCL plotting scripts to be able to plot grids with the latest version of the workflow.

## TESTS CONDUCTED: 
On hera, ran tests with all three grids with the GFS_v15p2 and RRFS_v1beta suites (a total of 6 tests).  All were successful.

* Remove redundant model_configure.${CCPP_PHYS_SUITE} template files; use Jinja2 to create model_configure (#321)

## DESCRIPTION OF CHANGES:
* Remove model_configure template files whose names depend on the physics suite, i.e. files with names of the form model_configure.${CCPP_PHYS_SUITE}.  Only a single template file is needed because the contents of the model_configure file are not suite dependent.  This leaves just one template file (named model_configure).
* Change the function create_model_configure_file.sh and the template file model_configure so they use jinja2 instead of sed to replace placeholder values.
* Absorb the contents of the write-component template files wrtcmp_lambert_conformal, wrtcmp_regional_latlon, and wrtcmp_rotated_latlon into the new jinja2-compliant model_configure file.  We can do this because Jinja2 allows use of if-statements in the template file.
* In the new model_configure jinja2 template file, include comments to explain the various write-component parameters.

## TESTS CONDUCTED:
On Hera, ran the two WE2E tests new_ESGgrid and new_GFDLgrid.  The first uses a "lambert_conformal" type of write-component grid, and the second uses a "rotated_latlon" type of write-component grid.  (The write-component also allows "regional_latlon" type grids, which is just the usual earth-relative latlon coordinate system, but we do not have any cases that use that.)  Both tests succeeded.

## ISSUE (optional): 
This PR resolves issue #281.

* Add Thompson ice- and water-friendly aerosol climo file support (#332)

* Add if statement in set_thompson_mp_fix_files.sh to source Thompson climo file when using a combination of a Thompson-based SDF and non-RAP/HRRR external model data

* Modify if statement based on external models for Thompson climo file

* Remove workflow variable EMC_GRID_NAME (#333)

## DESCRIPTION OF CHANGES: 
* Remove the workflow variable EMC_GRID_NAME.  Henceforth, PREDEF_GRID_NAME is the only variable that can be used to set the name of the predefined grid to use.
* Make appropriate change of variable name (EMC_GRID_NAME --> PREDEF_GRID_NAME) in the WE2E test configuration files.
* Change anywhere the "conus" and "conus_c96" grids are specified to "EMC_CONUS_3km" and "EMC_CONUS_coarse", respectively.
* Rename WE2E test configuration files with names containing the strings "conus" and "conus_c96" by replacing these strings with "EMC_CONUS_3km" and "EMC_CONUS_coarse", respectively.
* Update the list of WE2E test names (tests/baselines_list.txt).
* Bug fixes not directly related to grids:
  * In config.nco.sh, remove settings of QUEUE_DEFAULT, QUEUE_HPSS, and QUEUE_FCST since these are now set automatically (due to another PR).
  * In the template file FV3LAM_wflow.xml, add the ensemble member name after RUN_FCST_TN in the dependency of the run_post metatask.

## TESTS CONDUCTED: 
Since this change only affects runs in NCO mode, the following NCO-mode WE2E tests were rerun on hera, all successfully:
```
nco_EMC_CONUS_3km                                       SUCCESS
nco_EMC_CONUS_coarse                                    SUCCESS
nco_EMC_CONUS_coarse__suite_FV3_GFS_2017_gfdlmp         SUCCESS
nco_RRFS_CONUS_25km_HRRRX_RAPX                          SUCCESS
nco_RRFS_CONUS_3km_FV3GFS_FV3GFS                        SUCCESS
nco_RRFS_CONUS_3km_HRRRX_RAPX                           SUCCESS
nco_ensemble                                            SUCCESS
```

* Port workflow to Orion (#309)

## DESCRIPTION OF CHANGES:
* Add stanzas for Orion where necessary.
* Add new module files for Orion.
* On Orion, both the slurm partition and the slurm QOS need to be specified in the rocoto XML in order to be able to have wall times longer than 30 mins (the partition needs to be specified because it is by default "debug", which has a limit of 30 mins).  Thus, introduce modifications to more easily specify slurm partitions:
    * Remove the workflow variables QUEUE_DEFAULT_TAG, QUEUE_HPSS_TAG, and QUEUE_FCST_TAG that are currently used to determine whether QUEUE_DEFAULT, QUEUE_HPSS, and QUEUE_FCST specify the names of queue/QOS's or slurm partitions.
    * Add the workflow variables PARTITION_DEFAULT_TAG, PARTITION_HPSS_TAG, and PARTITION_FCST_TAG.  These will be used to specify slurm partitions only, and the variables QUEUE_DEFAULT, QUEUE_HPSS, and QUEUE_FCST will be used to specify queues/QOS's only.

IMPORTANT NOTE:
On Orion, in order to load the regional_workflow environment needed for generating an experiment, the user must first issue the following commands:
```
module use -a /apps/contrib/miniconda3-noaa-gsl/modulefiles
module load miniconda3
conda activate regional_workflow
```

## TESTS CONDUCTED:
Ran 11 WE2E tests on Orion, Hera, and Cheyenne.

Results on Orion:
```
community_ensemble_2mems          SUCCESS
DOT_OR_USCORE                     SUCCESS
grid_GSD_HRRR_AK_50km             FAILURE - In the run_fcst task.
  * Error message:
  !!! (1) Error in subr radiation_aerosols: unrealistic surface pressure =
           1                     NaN
new_ESGgrid                       SUCCESS
new_GFDLgrid                      SUCCESS
regional_001                      SUCCESS
regional_002                      SUCCESS
suite_FV3_GFS_v15p2               SUCCESS
suite_FV3_GFS_v16beta             SUCCESS
suite_FV3_GSD_SAR                 SUCCESS
suite_FV3_GSD_v0                  SUCCESS
```
Results on Hera:
```
community_ensemble_2mems          SUCCESS
DOT_OR_USCORE                     SUCCESS
grid_GSD_HRRR_AK_50km             SUCCESS
new_ESGgrid                       SUCCESS
new_GFDLgrid                      SUCCESS
regional_001                      SUCCESS
regional_002                      SUCCESS
suite_FV3_GFS_v15p2               SUCCESS
suite_FV3_GFS_v16beta             SUCCESS
suite_FV3_GSD_SAR                 SUCCESS
suite_FV3_GSD_v0                  SUCCESS
```
Results on Cheyenne:
```
community_ensemble_2mems          SUCCESS
DOT_OR_USCORE                     SUCCESS
grid_GSD_HRRR_AK_50km             FAILURE - In run_fcst task.
  * Error message:
  !!! (1) Error in subr radiation_aerosols: unrealistic surface pressure =
           1                     NaN
new_ESGgrid                       SUCCESS
new_GFDLgrid                      SUCCESS
regional_001                      SUCCESS
regional_002                      SUCCESS
suite_FV3_GFS_v15p2               SUCCESS
suite_FV3_GFS_v16beta             SUCCESS
suite_FV3_GSD_SAR                 SUCCESS
suite_FV3_GSD_v0                  SUCCESS
```
All succeed except GSD_HRRR_AK_50km on Orion and Cheyenne.  It is not clear why grid_GSD_HRRR_AK_50km fails on Orion and Cheyenne but not Hera.  Seems to point to a bug in the forecast model.  These two failures are not so important since this grid will soon be deprecated.

Also tested successfully on Jet by @JeffBeck-NOAA and on Odin and Stampede by @ywangwof.

## ISSUE:
This resolves Issue #152.

## CONTRIBUTORS:
@JeffBeck-NOAA @ywangwof @christinaholtNOAA

* Removed comments from exregional_get_mrms_files.sh and removed fhzero from FV3.input.yml

* Update FV3.input.nml for fhzero = 1.0

* Updated conf files for file name conventions.

* Updated MET scripts and MRMS pull scripts.

* Removed comments from exregional_get_mrms_files.sh and removed fhzero from FV3.input.yml

Co-authored-by: gsketefian <[email protected]>
Co-authored-by: Michael Kavulich <[email protected]>
Co-authored-by: JeffBeck-NOAA <[email protected]>
Co-authored-by: Jamie Wolff <[email protected]>

* Change cov_thresh for REFL to be a true max in nbrhood as SPC does.

* Separated Pull Data Scripts from Run Vx Scripts: Feature/add_metplus (#2)

* Job script for get_obs_ccpa

* Jobs script for get_obs_mrms

* Jobs script for get_obs_ndas

* Added external variables necessary to get_ccpa script

* Updated workflow template with separate get obs tasks

* Separated pull scripts from run scripts

* Added necessary defaults/values for defining pull tasks

* Added module files, default config.sh options, and changed dependencies for vx tasks

* Changed name of new workflow to FV3LAM_wflow.xml

* Added task get_obs_tn, removed config.sh, updated config_defaults and config.community.sh

* Adjusted the community and default config files based on comments

* Updated FV3LAM workflow

* Fixed discrepancies in config.community.sh

* Fixed discrepancies in config_defaults.sh

* Fixed discrepancies in config_defaults.sh round 2

* Fixed discrepancies in config_defaults.sh round 3

* Fixed discrepancies in config_defaults.sh round 4

* Fixed discrepancies in config.community.sh round 2

* Fixed discrepancies in config.community.sh round 3

* Fixed discrepancies in generate_FV3LAM_wflow.sh

* Fixed discrepancies in generate_FV3LAM_wflow.sh round 2

* Fixed discrepancies in generate_FV3LAM_wflow.sh round 3

* Updated FV3LAM_wflow template

* Fixed Vx Task Dependencies in Workflow: Feature/add metplus (#3)

* Job script for get_obs_ccpa

* Jobs script for get_obs_mrms

* Jobs script for get_obs_ndas

* Added external variables necessary to get_ccpa script

* Updated workflow template with separate get obs tasks

* Separated pull scripts from run scripts

* Added necessary defaults/values for defining pull tasks

* Added module files, default config.sh options, and changed dependencies for vx tasks

* Changed name of new workflow to FV3LAM_wflow.xml

* Added task get_obs_tn, removed config.sh, updated config_defaults and config.community.sh

* Adjusted the community and default config files based on comments

* Updated FV3LAM workflow

* Fixed discrepancies in config.community.sh

* Fixed discrepancies in config_defaults.sh

* Fixed discrepancies in config_defaults.sh round 2

* Fixed discrepancies in config_defaults.sh round 3

* Fixed discrepancies in config_defaults.sh round 4

* Fixed discrepancies in config.community.sh round 2

* Fixed discrepancies in config.community.sh round 3

* Fixed discrepancies in generate_FV3LAM_wflow.sh

* Fixed discrepancies in generate_FV3LAM_wflow.sh round 2

* Fixed discrepancies in generate_FV3LAM_wflow.sh round 3

* Updated FV3LAM_wflow template

* Fixed the dependencies of the vx tasks

* Manual merge with develop that didn't seem to work before. Trying to get feature branch updated so it will run again!

* Add local module files

* Add environment variable for SCRIPTSDIR

* Remove echo statement

* Remove old module files

* Update to config_default for walltime for ndas pull. Update to metplus parm for obs file template. Update to FV3LAM xml to not include 00 hour for verification

* Update template to remove full path

* Verification channges for obs. (#4)

* Verification channges for obs.

* Update config_defaults.sh for vx description

* Update config_defaults.sh to remove extraneous MET info.

Co-authored-by: Michelle Harrold <[email protected]>

* Pull in updates from develop that were not merging properly. Small change to config.community to turn off vx tasks by default.

* Did manual merge of these files because it was not handled properly automatically

* Adding additional variables to METplus for regional workflow (#5)

* Updates to address comments in PR review

* Changed name of get_obs to remove _tn

* Missed removal of on get_obs

Co-authored-by: michelleharrold <[email protected]>
Co-authored-by: gsketefian <[email protected]>
Co-authored-by: Michael Kavulich <[email protected]>
Co-authored-by: JeffBeck-NOAA <[email protected]>
Co-authored-by: Lindsay <[email protected]>
Co-authored-by: Michelle Harrold <[email protected]>
Co-authored-by: PerryShafran-NOAA <[email protected]>
epic-cicd-jenkins pushed a commit that referenced this issue Apr 17, 2023
* Fix to post flat file.

* Create MET and METplus config files under ush/templates/parm

* Added script to pull and reorg ccpa data. Added a script to run gridstat with METplus. Updated MET and METplus config files.

* Added new jjob for running grid-stat vx. Updated setup.sh to include grid-stat vx. Updated run_gridstatvx script.

* Fixed typo on script name from ksh to sh

* Moved some hard coded items out from the script to the XML

* Updates to get METplus to run with fewer hard-coded paths.

* Updates to add grid-stat task to XML generation.

* Bug fixes for adding grid-stat to XML generation

* Updates to remove hard-coded paths in config files

* Change log dir to put master_metplus log file with other logs under log/, rather than default logs/.

* Updates to generate xml without hard-coded paths for MET

* Add hera gridstat module file

* Add METplus point-stat task for both sfc and upper air

* Small tweaks to remove hard coded paths and add some flexibility

* Updates for adding point-stat into auto-generated xml

* Add in function to set point-stat task to FALSE

* Final tweaks to get it to generate the xml correctly

* Minor updates to run ensure 0,6,12,18

* Tweaks to var list for Point-Stat

* Add METplus settings to config_defaults

* Move quote for end of settings and fix extra comment.

* Fix typos to populate templates correctly

* Updated to include SCRIPTSDIR and other MET specific settings along with updates to FHR syntax

* Update module loads on hera

* Fixed comment for BOTH_VARn_THRESH to avoid syntax issues

* Added files to run grid_stat for a variety of accumulation intervals, including 3, 6, and 24h

* Added module load hpss

* Remove module load informatino from these scripts

* Updated the method of turning on/off vx tasks using jinja template if statement

* Remove commented out lines of code. Fixed typo. Removed gen_wflow.out file.

* Updated pull scripts to have file names dependent on date to pull from HPSS. Updated to export a few more local variables that METplus conf needed in scripts. Updated workflow to use service queue (for now) to for 1h grid_stat and point_stat run and default for 3+h accumulation grid_stat runs)

* moved common_hera.conf to common.conf - no platform specific information included that needs to be handled.

* Remove common_hera.conf

* Add scripts to pull and process MRMS data from NOAA HPSS

* Updates for REFC vx tasks

* updates to obs pull scripts

* Update for adding in reflectivity verification using MRMS analyses and updating name of model output to RRFS rather than HRRR

* Updates to account for CCPA issues on HPSS - day off for 00-05 UTC directories

* Verification mods to feature/add metplus (#1)

* Remove unused/outdated code (#313)

## DESCRIPTION OF CHANGES:
* In setup.sh and generate_FV3LAM_wflow.sh, remove temporary codes that fix bugs in the FV3_GFS_2017_gfdlmp_regional suite definition file because those bugs have been fixed (in the ufs-weather-model repo).
* In setup.sh, remove block of code that is no longer necessary because chgres_cube can now initialize from external model data with either 4 or 9 soil levels, and run with LSMs of either 4 or 9 soil levels.
* Remove modifications to LD_LIBRARY_PATH in exregional_run_fcst.sh.
* For the make_ics and make_lbcs tasks, move the setting of APRUN and other machine-specific actions from the J-job to the ex-script in order to be consistent with the other workflow tasks.
* Fix indentation and edit comments.
* Remove unused file load_fv3gfs_modules.sh.

## TESTS CONDUCTED: 
Ran two WE2E tests on hera, new_ESGgrid and new_GFDLgrid:
* new_ESGgrid uses the FV3_GFS_2017_gfdlmp_regional suite.  The test was successful.
* new_GFDLgrid uses the FV3_GFS_2017_gfdlmp suite.  The test was successful.

## ISSUE (optional): 
This resolves issue #198.

* Add and call a function that checks for use of Thompson microphysics parameterization in the SDF and if so, adjusts certain workflow arrays to contain the names and other associated values of the fixed files needed by this parameterization so that those files are automatically copied and/or linked to. (#319)

## DESCRIPTION OF CHANGES: 
Add and call a function that checks for use of Thompson microphysics parameterization in the suite definition file (SDF).  If not, do nothing.  If so, add to the appropriate workflow arrays the names and other associated values of the fixed files needed by this parameterization so that they are automatically copied and/or linked to instead of being regenerated from scratch in the run_fcst task.

## TESTS CONDUCTED: 
On hera, ran two WE2E tests, one in NCO mode (nco_RRFS_CONUS_25km_HRRRX_RAPX) and the other in community mode (suite_FV3_GSD_v0).  These use suites FV3_GSD_SAR and FV3_GSD_v0, respectively, and both of these call Thompson microphysics.  Both succeeded.

## ISSUE (optional):
This PR resolves issue #297.

* RRFS_v1beta SDF changes after reverting from GSL to GFS GWD suite (#322) (#327)

## DESCRIPTION OF CHANGES:
Removed checks on the RRFS_v1beta SDF implemented for use with the GSL GWD suite (now uses the GFS GWD suite).  No longer copies staged orography files necessary for the GSL GWD suite.

## TESTS CONDUCTED:
Runs to completion on Hera. End-to-end runs DOT_OR_USCORE and suite_FV3_RRFS_v1beta succeeded on Cheyenne.

Co-authored-by: JeffBeck-NOAA <[email protected]>

* Update FV3.input.nml for fhzero = 1.0

* Updated conf files for file name conventions.

* Updated MET scripts and MRMS pull scripts.

* Adjust RRFS_CONUS_... grids (#294)

## DESCRIPTION OF CHANGES: 
* Adjust RRFS_CONUS_25km, RRFS_CONUS_13km, and RRFS_CONUS_3km grid parameters so that:
  * All grids, including their 4-cell-wide halos, lie completely within the HRRRX domain.
  * All grids have dimensions nx and ny that factor "nicely", i.e. they don't have factors greather than 7.
  * The write-component grids corresponding to these three native grids cover as much of the native grids as possible without going outside of the native grid boundaries.  The updated NCL scripts (see below) were used to generate the write-component grid parameters.
* For the RRFS_CONUS_13km grid, reduce the time step (DT_ATMOS) from 180sec to 45sec.  This is necessary to get a successful forecast with the GSD_SAR suite, and thus likely also the RRFS_v1beta suite.
* Modify WE2E testing system as follows:
  * Add new tests with the RRFS_CONUS_25km, RRFS_CONUS_13km, and RRFS_CONUS_3km grids that use the GFS_v15p2 and RRFS_v1beta suites (which are now the ones officially supported in the first release of the short-range weather app) instead of the GFS_v16beta and GSD_SAR suites, respectively.
  * For clarity, rename the test configuration files that use the GFS_v16beta and GSD_SAR suites so they include the suite name.
  * Update list of WE2E tests (baselines_list.txt).
* Update the NCL plotting scripts to be able to plot grids with the latest version of the workflow.

## TESTS CONDUCTED: 
On hera, ran tests with all three grids with the GFS_v15p2 and RRFS_v1beta suites (a total of 6 tests).  All were successful.

* Remove redundant model_configure.${CCPP_PHYS_SUITE} template files; use Jinja2 to create model_configure (#321)

## DESCRIPTION OF CHANGES:
* Remove model_configure template files whose names depend on the physics suite, i.e. files with names of the form model_configure.${CCPP_PHYS_SUITE}.  Only a single template file is needed because the contents of the model_configure file are not suite dependent.  This leaves just one template file (named model_configure).
* Change the function create_model_configure_file.sh and the template file model_configure so they use jinja2 instead of sed to replace placeholder values.
* Absorb the contents of the write-component template files wrtcmp_lambert_conformal, wrtcmp_regional_latlon, and wrtcmp_rotated_latlon into the new jinja2-compliant model_configure file.  We can do this because Jinja2 allows use of if-statements in the template file.
* In the new model_configure jinja2 template file, include comments to explain the various write-component parameters.

## TESTS CONDUCTED:
On Hera, ran the two WE2E tests new_ESGgrid and new_GFDLgrid.  The first uses a "lambert_conformal" type of write-component grid, and the second uses a "rotated_latlon" type of write-component grid.  (The write-component also allows "regional_latlon" type grids, which is just the usual earth-relative latlon coordinate system, but we do not have any cases that use that.)  Both tests succeeded.

## ISSUE (optional): 
This PR resolves issue #281.

* Add Thompson ice- and water-friendly aerosol climo file support (#332)

* Add if statement in set_thompson_mp_fix_files.sh to source Thompson climo file when using a combination of a Thompson-based SDF and non-RAP/HRRR external model data

* Modify if statement based on external models for Thompson climo file

* Remove workflow variable EMC_GRID_NAME (#333)

## DESCRIPTION OF CHANGES: 
* Remove the workflow variable EMC_GRID_NAME.  Henceforth, PREDEF_GRID_NAME is the only variable that can be used to set the name of the predefined grid to use.
* Make appropriate change of variable name (EMC_GRID_NAME --> PREDEF_GRID_NAME) in the WE2E test configuration files.
* Change anywhere the "conus" and "conus_c96" grids are specified to "EMC_CONUS_3km" and "EMC_CONUS_coarse", respectively.
* Rename WE2E test configuration files with names containing the strings "conus" and "conus_c96" by replacing these strings with "EMC_CONUS_3km" and "EMC_CONUS_coarse", respectively.
* Update the list of WE2E test names (tests/baselines_list.txt).
* Bug fixes not directly related to grids:
  * In config.nco.sh, remove settings of QUEUE_DEFAULT, QUEUE_HPSS, and QUEUE_FCST since these are now set automatically (due to another PR).
  * In the template file FV3LAM_wflow.xml, add the ensemble member name after RUN_FCST_TN in the dependency of the run_post metatask.

## TESTS CONDUCTED: 
Since this change only affects runs in NCO mode, the following NCO-mode WE2E tests were rerun on hera, all successfully:
```
nco_EMC_CONUS_3km                                       SUCCESS
nco_EMC_CONUS_coarse                                    SUCCESS
nco_EMC_CONUS_coarse__suite_FV3_GFS_2017_gfdlmp         SUCCESS
nco_RRFS_CONUS_25km_HRRRX_RAPX                          SUCCESS
nco_RRFS_CONUS_3km_FV3GFS_FV3GFS                        SUCCESS
nco_RRFS_CONUS_3km_HRRRX_RAPX                           SUCCESS
nco_ensemble                                            SUCCESS
```

* Port workflow to Orion (#309)

## DESCRIPTION OF CHANGES:
* Add stanzas for Orion where necessary.
* Add new module files for Orion.
* On Orion, both the slurm partition and the slurm QOS need to be specified in the rocoto XML in order to be able to have wall times longer than 30 mins (the partition needs to be specified because it is by default "debug", which has a limit of 30 mins).  Thus, introduce modifications to more easily specify slurm partitions:
    * Remove the workflow variables QUEUE_DEFAULT_TAG, QUEUE_HPSS_TAG, and QUEUE_FCST_TAG that are currently used to determine whether QUEUE_DEFAULT, QUEUE_HPSS, and QUEUE_FCST specify the names of queue/QOS's or slurm partitions.
    * Add the workflow variables PARTITION_DEFAULT_TAG, PARTITION_HPSS_TAG, and PARTITION_FCST_TAG.  These will be used to specify slurm partitions only, and the variables QUEUE_DEFAULT, QUEUE_HPSS, and QUEUE_FCST will be used to specify queues/QOS's only.

IMPORTANT NOTE:
On Orion, in order to load the regional_workflow environment needed for generating an experiment, the user must first issue the following commands:
```
module use -a /apps/contrib/miniconda3-noaa-gsl/modulefiles
module load miniconda3
conda activate regional_workflow
```

## TESTS CONDUCTED:
Ran 11 WE2E tests on Orion, Hera, and Cheyenne.

Results on Orion:
```
community_ensemble_2mems          SUCCESS
DOT_OR_USCORE                     SUCCESS
grid_GSD_HRRR_AK_50km             FAILURE - In the run_fcst task.
  * Error message:
  !!! (1) Error in subr radiation_aerosols: unrealistic surface pressure =
           1                     NaN
new_ESGgrid                       SUCCESS
new_GFDLgrid                      SUCCESS
regional_001                      SUCCESS
regional_002                      SUCCESS
suite_FV3_GFS_v15p2               SUCCESS
suite_FV3_GFS_v16beta             SUCCESS
suite_FV3_GSD_SAR                 SUCCESS
suite_FV3_GSD_v0                  SUCCESS
```
Results on Hera:
```
community_ensemble_2mems          SUCCESS
DOT_OR_USCORE                     SUCCESS
grid_GSD_HRRR_AK_50km             SUCCESS
new_ESGgrid                       SUCCESS
new_GFDLgrid                      SUCCESS
regional_001                      SUCCESS
regional_002                      SUCCESS
suite_FV3_GFS_v15p2               SUCCESS
suite_FV3_GFS_v16beta             SUCCESS
suite_FV3_GSD_SAR                 SUCCESS
suite_FV3_GSD_v0                  SUCCESS
```
Results on Cheyenne:
```
community_ensemble_2mems          SUCCESS
DOT_OR_USCORE                     SUCCESS
grid_GSD_HRRR_AK_50km             FAILURE - In run_fcst task.
  * Error message:
  !!! (1) Error in subr radiation_aerosols: unrealistic surface pressure =
           1                     NaN
new_ESGgrid                       SUCCESS
new_GFDLgrid                      SUCCESS
regional_001                      SUCCESS
regional_002                      SUCCESS
suite_FV3_GFS_v15p2               SUCCESS
suite_FV3_GFS_v16beta             SUCCESS
suite_FV3_GSD_SAR                 SUCCESS
suite_FV3_GSD_v0                  SUCCESS
```
All succeed except GSD_HRRR_AK_50km on Orion and Cheyenne.  It is not clear why grid_GSD_HRRR_AK_50km fails on Orion and Cheyenne but not Hera.  Seems to point to a bug in the forecast model.  These two failures are not so important since this grid will soon be deprecated.

Also tested successfully on Jet by @JeffBeck-NOAA and on Odin and Stampede by @ywangwof.

## ISSUE:
This resolves Issue #152.

## CONTRIBUTORS:
@JeffBeck-NOAA @ywangwof @christinaholtNOAA

* Removed comments from exregional_get_mrms_files.sh and removed fhzero from FV3.input.yml

* Update FV3.input.nml for fhzero = 1.0

* Updated conf files for file name conventions.

* Updated MET scripts and MRMS pull scripts.

* Removed comments from exregional_get_mrms_files.sh and removed fhzero from FV3.input.yml

Co-authored-by: gsketefian <[email protected]>
Co-authored-by: Michael Kavulich <[email protected]>
Co-authored-by: JeffBeck-NOAA <[email protected]>
Co-authored-by: Jamie Wolff <[email protected]>

* Change cov_thresh for REFL to be a true max in nbrhood as SPC does.

* Job script for get_obs_ccpa

* Jobs script for get_obs_mrms

* Jobs script for get_obs_ndas

* Added external variables necessary to get_ccpa script

* Updated workflow template with separate get obs tasks

* Separated pull scripts from run scripts

* Added necessary defaults/values for defining pull tasks

* Added module files, default config.sh options, and changed dependencies for vx tasks

* Changed name of new workflow to FV3LAM_wflow.xml

* Added task get_obs_tn, removed config.sh, updated config_defaults and config.community.sh

* Adjusted the community and default config files based on comments

* Updated FV3LAM workflow

* Fixed discrepancies in config.community.sh

* Fixed discrepancies in config_defaults.sh

* Fixed discrepancies in config_defaults.sh round 2

* Fixed discrepancies in config_defaults.sh round 3

* Fixed discrepancies in config_defaults.sh round 4

* Fixed discrepancies in config.community.sh round 2

* Fixed discrepancies in config.community.sh round 3

* Fixed discrepancies in generate_FV3LAM_wflow.sh

* Fixed discrepancies in generate_FV3LAM_wflow.sh round 2

* Fixed discrepancies in generate_FV3LAM_wflow.sh round 3

* Updated FV3LAM_wflow template

* Separated Pull Data Scripts from Run Vx Scripts: Feature/add_metplus (#2)

* Job script for get_obs_ccpa

* Jobs script for get_obs_mrms

* Jobs script for get_obs_ndas

* Added external variables necessary to get_ccpa script

* Updated workflow template with separate get obs tasks

* Separated pull scripts from run scripts

* Added necessary defaults/values for defining pull tasks

* Added module files, default config.sh options, and changed dependencies for vx tasks

* Changed name of new workflow to FV3LAM_wflow.xml

* Added task get_obs_tn, removed config.sh, updated config_defaults and config.community.sh

* Adjusted the community and default config files based on comments

* Updated FV3LAM workflow

* Fixed discrepancies in config.community.sh

* Fixed discrepancies in config_defaults.sh

* Fixed discrepancies in config_defaults.sh round 2

* Fixed discrepancies in config_defaults.sh round 3

* Fixed discrepancies in config_defaults.sh round 4

* Fixed discrepancies in config.community.sh round 2

* Fixed discrepancies in config.community.sh round 3

* Fixed discrepancies in generate_FV3LAM_wflow.sh

* Fixed discrepancies in generate_FV3LAM_wflow.sh round 2

* Fixed discrepancies in generate_FV3LAM_wflow.sh round 3

* Updated FV3LAM_wflow template

* Fixed the dependencies of the vx tasks

* Fixed Vx Task Dependencies in Workflow: Feature/add metplus (#3)

* Job script for get_obs_ccpa

* Jobs script for get_obs_mrms

* Jobs script for get_obs_ndas

* Added external variables necessary to get_ccpa script

* Updated workflow template with separate get obs tasks

* Separated pull scripts from run scripts

* Added necessary defaults/values for defining pull tasks

* Added module files, default config.sh options, and changed dependencies for vx tasks

* Changed name of new workflow to FV3LAM_wflow.xml

* Added task get_obs_tn, removed config.sh, updated config_defaults and config.community.sh

* Adjusted the community and default config files based on comments

* Updated FV3LAM workflow

* Fixed discrepancies in config.community.sh

* Fixed discrepancies in config_defaults.sh

* Fixed discrepancies in config_defaults.sh round 2

* Fixed discrepancies in config_defaults.sh round 3

* Fixed discrepancies in config_defaults.sh round 4

* Fixed discrepancies in config.community.sh round 2

* Fixed discrepancies in config.community.sh round 3

* Fixed discrepancies in generate_FV3LAM_wflow.sh

* Fixed discrepancies in generate_FV3LAM_wflow.sh round 2

* Fixed discrepancies in generate_FV3LAM_wflow.sh round 3

* Updated FV3LAM_wflow template

* Fixed the dependencies of the vx tasks

* Manual merge with develop that didn't seem to work before. Trying to get feature branch updated so it will run again!

* Add local module files

* Add environment variable for SCRIPTSDIR

* Remove echo statement

* Remove old module files

* Update to config_default for walltime for ndas pull. Update to metplus parm for obs file template. Update to FV3LAM xml to not include 00 hour for verification

* Update template to remove full path

* Verification channges for obs. (#4)

* Verification channges for obs.

* Update config_defaults.sh for vx description

* Update config_defaults.sh to remove extraneous MET info.

Co-authored-by: Michelle Harrold <[email protected]>

* Initial METplus .confs and MET config files for EnsembleStat APCP

* J-Job script for running ensemble stat

* Exregional script for ensemble-stat

* Added EnsembleStat.conf for A6 and A24. Added PCPCombine to A3, A6, and A24.

* Added EnsembleStatConfig files for 6 and 24h

* Copy of workflow template with precipitation ensemble tasks added. Will become main template when testing is complete

* Added export statement for number of ensemble members

* Added necessary task definitions in ush

* Updated workflow to included ENTITY definitions for ensstat

* Fixed typo

* Added ens vx configs

* Pull in updates from develop that were not merging properly. Small change to config.community to turn off vx tasks by default.

* Added/mod files for point ens vx.

* Updated metplus conf files for ens point vx

* Did manual merge of these files because it was not handled properly automatically

* Adding additional variables to METplus for regional workflow (#5)

* Changes made based on meeting with Michelle and Jamie

* Updating fork

* Cleanup after merge

* Added additional ens vx

* Ensemble point vx mods

* Additional updates for ens and det vx

* ensgrid_mean and ensgrid_prob .conf files for APCP

* Updates for ensemble vx.

* Added mean and prob point-stat configs

* Updates to ensgrid_vx

* Updates for mean/prob vx.

* Updates to FV3LAM_wflow.xml

* Deterministic and ensembel vx updates.

* Ensgrid mean

* Update setup.sh

* Changed workflow template title

* Updates to deterministic and ensemble verification

* Created EnsembleStat METplus conf and MET config files for REFC

* Added reflectivity mean and prob METplus and MET config files. Updated APCP mean and prob METplus and MET config files.

* Added all J-job scripts, exregional scripts, and necessary definitons for workflow generation for all ensgrid_mean and ensgrid_prob tasks

* Updates to workflow to add ensgrid_vx

* Changes I made to account for runtime errors.

* Made changes to directory structures

* Made changes to directory structures and variables

* Changed log files and stage dir.

* Changes for grid- and point-vx.

* Updated METplus ensemble precip conf files.

* Mods for ensemble and deterministic vx.

* Change to GridStatConfig_REFC_mean

* Updated EnsembleStat_REFC.conf

* Updated to METv10.0.0

* Updated conf files for paths.

* Updated FV3LAM_wflow.xml template.

* Mods for vx dependencies

* Updated for censor thresh in METplus conf files; changes to FV3LAM_wflow.xml after sync with develop.

* Updated exregional_run_fcst.sh generate_FV3LAM_wflow.sh to address merge with develop.

* Mods for ensemble precip vx, handling padded/non-padded ensemble member names, fixes for python environment for obs pull.

* Changes to RETOP (units) and REFC (naming and level) verification.

* Fix OUTPUT_BASE for deterministic vx.

* Changes to some verification ex-scripts for syntax and path fixes. Included start end dates of incorrect 01-h CCPA data. Removed some extra lines in FV3LAM_wflow.xml template.

* Changed comp. ref. variable name in GridStat_REFC_prob.conf

* Changed comp. ref. level in GridStat_REFC_prob.conf

* Updated logic for number padding in the directory name when running in ensemble mode.

* Added MET ensemble vx WE2E test.

* Modified location of obs to live outside cycle dir, allowing for obs to be shared across cycles.

* Mods to address comments on PR575.

* Updated ensemble METPlus conf files for changes to post output name.

* Addessed comments in PR and mods for 10-m WIND.

* Addressing final comments in PR.

Co-authored-by: Jamie Wolff <[email protected]>
Co-authored-by: gsketefian <[email protected]>
Co-authored-by: Michael Kavulich <[email protected]>
Co-authored-by: JeffBeck-NOAA <[email protected]>
Co-authored-by: lindsayrblank <[email protected]>
Co-authored-by: Michelle Harrold <[email protected]>
Co-authored-by: PerryShafran-NOAA <[email protected]>
AlexanderRichert-NOAA pushed a commit to AlexanderRichert-NOAA/ufs-weather-model that referenced this issue Jul 7, 2023
…odules

update acorn to spack-stack 1.4.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants