RFC 13303 - place PlanetiQ RO data in monitor mode

On WCOSS2, upgraded the GFS to v16.3.19. The reason for this change is the following: provider PlanetiQ offers two versions of their data: basic and premium. Premium is high quality data. Basic is a degraded version of the premium data. EMC learned last week that NESDIS purchased the basic (degraded) data. This ARFC prevents the GFS from assimilating the degraded PlanetiQ GPS Radio Occultation (GPS-RO) data when it flows into the GFS. This update also increased the walltime limit for the jenkfgdas_update job from 30 min to 40 min to address runtime variation.

Implemented on September 30 from 1330Z to 1615Z.

What's Changed

GFSv16.3.19 - place PlanetiQ RO data in monitor mode by @KateFriedman-NOAA in #2968

Full Changelog: gfs.v16.3.18...gfs.v16.3.19

Supported resolutions and platforms

WCOSS2 only

• C768C384L127 cycled & C768 forecast-only
• C384C192L127 cycled & C384 forecast-only
• C192C96L127 cycled & C192 forecast-only

Supported verification packages

• Fit2Obs v1.5
• METplus EMC_verif-global.v2.10.0

Setup steps

1. Quick clone and build instructions

git clone -b gfs.v16.3.19 https://github.com/NOAA-EMC/global-workflow.git
cd global-workflow/sorc
sh checkout.sh
sh build_all.sh
sh link_fv3gfs.sh emc wcoss2

Follow additional build and link instructions here: https://github.com/NOAA-EMC/global-workflow/wiki/Clone-and-build-global-workflow/08b2ec4d0956ea1936d567d9a1ac32614aedfec1

2. Run experiment setup scripts

See the following global-workflow wiki page: https://github.com/NOAA-EMC/global-workflow/wiki/Run-setup-scripts-to-generate-experiment/c85d5c2f87b298b6f0536a2fc2fb425a092c70fc

Note: the above wiki links are to older revisions of the global-workflow GitHub wiki (before updates were made for GFSv17/develop). Use wcoss2 for the machine where needed (e.g. link_fv3gfs.sh script) and ignore references to now unsupported machines (e.g. WCOSS-Dell and WCOSS-Cray).

Report any issues to @KateFriedman-NOAA

What's Changed

GFSv16.3.18 - Increase the wait time for UKMet data in the WAFS blending_0p25 job by @KateFriedman-NOAA in #2933

Full Changelog: gfs.v16.3.17...gfs.v16.3.18

Supported resolutions and platforms

WCOSS2 only

• C768C384L127 cycled & C768 forecast-only
• C384C192L127 cycled & C384 forecast-only
• C192C96L127 cycled & C192 forecast-only

Supported verification packages

• Fit2Obs v1.5
• METplus EMC_verif-global.v2.10.0

Setup steps

1. Quick clone and build instructions

git clone -b gfs.v16.3.18 https://github.com/NOAA-EMC/global-workflow.git
cd global-workflow/sorc
sh checkout.sh
sh build_all.sh
sh link_fv3gfs.sh emc wcoss2

Follow additional build and link instructions here: https://github.com/NOAA-EMC/global-workflow/wiki/Clone-and-build-global-workflow/08b2ec4d0956ea1936d567d9a1ac32614aedfec1

2. Run experiment setup scripts

See the following global-workflow wiki page: https://github.com/NOAA-EMC/global-workflow/wiki/Run-setup-scripts-to-generate-experiment/c85d5c2f87b298b6f0536a2fc2fb425a092c70fc

Note: the above wiki links are to older revisions of the global-workflow GitHub wiki (before updates were made for GFSv17/develop). Use wcoss2 for the machine where needed (e.g. link_fv3gfs.sh script) and ignore references to now unsupported machines (e.g. WCOSS-Dell and WCOSS-Cray).

Report any issues to @KateFriedman-NOAA

What's Changed

Updates for the Real Time Ocean Forecast System (RTOFS) v2.4.4 operational upgrade (#2800) by @KateFriedman-NOAA in #2921

Full Changelog: gfs.v16.3.16.1...gfs.v16.3.17

Supported resolutions and platforms

WCOSS2 only

• C768C384L127 cycled & C768 forecast-only
• C384C192L127 cycled & C384 forecast-only
• C192C96L127 cycled & C192 forecast-only

Supported verification packages

• Fit2Obs v1.5
• METplus EMC_verif-global.v2.10.0

Setup steps

1. Quick clone and build instructions

git clone -b gfs.v16.3.17 https://github.com/NOAA-EMC/global-workflow.git
cd global-workflow/sorc
sh checkout.sh
sh build_all.sh
sh link_fv3gfs.sh emc wcoss2

Follow additional build and link instructions here: https://github.com/NOAA-EMC/global-workflow/wiki/Clone-and-build-global-workflow/08b2ec4d0956ea1936d567d9a1ac32614aedfec1

2. Run experiment setup scripts

See the following global-workflow wiki page: https://github.com/NOAA-EMC/global-workflow/wiki/Run-setup-scripts-to-generate-experiment/c85d5c2f87b298b6f0536a2fc2fb425a092c70fc

Note: the above wiki links are to older revisions of the global-workflow GitHub wiki (before updates were made for GFSv17/develop). Use wcoss2 for the machine where needed (e.g. link_fv3gfs.sh script) and ignore references to now unsupported machines (e.g. WCOSS-Dell and WCOSS-Cray).

Report any issues to @KateFriedman-NOAA

What's Changed

• Update to spack-stack 1.6.0 by @DavidHuber-NOAA in #2239
• Removes jlogfile and postmsg instances from workflow by @HenryRWinterbottom in #2312
• Jenkins updates by @TerrenceMcGuinness-NOAA in #2334
• Remove FIX* variables for fix subfolders by @KateFriedman-NOAA in #2337
• Update JEDI ATM to use .nc for obs and generalize x,y layout by @RussTreadon-NOAA in #2336
• Update ufs-waether-model hash by @JessicaMeixner-NOAA in #2338
• Rename the land DA jobs to snow DA to better reflect what they are doing by @CoryMartin-NOAA in #2330
• Fix rocoto forecast hour determination for GEFS by @WalterKolczynski-NOAA in #2351
• Move Jenkinsfile into ci subdirectory by @TerrenceMcGuinness-NOAA in #2355
• Turn on C48mx500_3DVarAOWCDA test on hera by @JessicaMeixner-NOAA in #2363
• Update stage IC to handle ocean perturbations by @NeilBarton-NOAA in #2364
• Sort list of coupler restart files for restart time determination by @DWesl in #2360
• Adding debug option for all build scripts by @HenryRWinterbottom in #2326
• Reduce Jenkins messaging to GitHub by @TerrenceMcGuinness-NOAA in #2370
• Reformat snowDA templates to jinja2 by @aerorahul in #2371
• Add Hercules support for the GSI monitor by @DavidHuber-NOAA in #2373
• Forgotten templated DO_VRFY_OCEANDA by @guillaumevernieres in #2379
• Clean out non-gfs top level variables by @KateFriedman-NOAA in #2366
• Add global-workflow infrastructure for ocean analysis recentering task by @AndrewEichmann-NOAA in #2299
• Add new BUFR table file parm/product/bufr_ij9km.txt for GFSv17 C1152 by @BoCui-NOAA in #2383
• Return ocnanalrun npes resource setting back to previous value by @AndrewEichmann-NOAA in #2386
• Rewrote pr_list_database.py to use wxflow's SQLiteDB Class by @TerrenceMcGuinness-NOAA in #2376
• Add switch to control debug=true on WCOSS2 for development testing by @KateFriedman-NOAA in #2388
• Add JEDI atmosphere only CI by @RussTreadon-NOAA in #2357
• Changed config.atmanl to allow non-hybrid background error yamls by @DavidNew-NOAA in #2394
• remove documentation about generating ICs using global-workflow by @aerorahul in #2397
• Fix KeyError issue in ocean/ice postprocessing job. by @aerorahul in #2398
• Missed a comma from the hotfix this AM by @aerorahul in #2399
• GitHub message error paths by @TerrenceMcGuinness-NOAA in #2401
• Fix ensemble archive groups to include all members by @CatherineThomas-NOAA in #2402
• Parse jediyaml only once by @aerorahul in #2387
• re-enable ci/cases/pr/C48mx500_3DVarAOWCDA.yaml by @JessicaMeixner-NOAA in #2405
• Add initial GSI-based soil analysis capability by @ClaraDraper-NOAA in #2263
• Add GEFS Ocean Perturbation Options by @NeilBarton-NOAA in #2385
• Updates to RTD documentation by @KateFriedman-NOAA in #2418
• Hotfix: send the correct number of build jobs for the UFS by @DavidHuber-NOAA in #2423
• Add WAFS jobs, scripts and ush to GFS v17 workflow repository by @YaliMao-NOAA in #2412
• hotfix to update full path to error logs on CI case fail by @TerrenceMcGuinness-NOAA in #2425
• Fix *earc jobs where the number of members isn't a multiple of 10 by @DavidHuber-NOAA in #2424
• Correct GDASApp paths by @RussTreadon-NOAA in #2435
• Add option to use traditional threading in the UFS by @aerorahul in #2384
• Uploading error logs to GitHub from Jenkins CI Runs by @TerrenceMcGuinness-NOAA in #2429
• Add a rotating subset of members for early-cycle enkf by @TravisElless-NOAA in #2427
• Update global-workflow and subcomponents to Hera/Rocky 8 partition by @HenryRWinterbottom in #2421
• Fix post log arg check and don't create build semaphore by @TerrenceMcGuinness-NOAA in #2440
• Modify APP based on RUN by @WalterKolczynski-NOAA in #2413
• Turn GEFS CI test on for Hera by @TerrenceMcGuinness-NOAA in #2442
• Refactor gempak jobs for new COM and style by @WalterKolczynski-NOAA in #2374
• Add GEFS ENS Atmos options by @NeilBarton-NOAA in #2392
• Archive the snow DA analysis into HPSS by @jiaruidong2017 in #2414
• Remove the reset of upper layer humidity by @CatherineThomas-NOAA in #2449
• Fix rotating member bugs by @TravisElless-NOAA in #2443
• Reenable the minimization monitor on Hera by @DavidHuber-NOAA in #2446
• Move bash utility functions out of preamble by @WalterKolczynski-NOAA in #2447
• Rename generate_com to declare_from_tmpl by @WalterKolczynski-NOAA in #2453
• Turn off reducedgrid in the EnKF by @CatherineThomas-NOAA in #2456
• The soca fix path is needed in config.prepoceanobs by @guillaumevernieres in #2460
• Add new data sources used in GFS v16.3.12 by @ADCollard in #2283
• Flip the build for GFS and GEFS with waves by @aerorahul in #2462
• Check the DO_JEDISNOWDA condition before adding snow DA analysis to the list by @jiaruidong2017 in #2471
• Improve error messaging to resolve bugzilla by @KateFriedman-NOAA in #2468
• Add support for CADS IR cloud detection scheme in the GSI by @wx20jjung in #2478
• Add oceanalecn to workflow generator by @AndrewEichmann-NOAA in #2409
• Fix paths in prepocnobs task by @AndrewEichmann-NOAA in #2459
• Update parm files for atm product by @WenMeng-NOAA in #2486
• Update fbwind for COM refactor by @WalterKolczynski-NOAA in #2479
• Upadte and tested CI Bash for WCOSS2 by @TerrenceMcGuinness-NOAA in #2481
• Add mean/spread for atmos grib2 by @WalterKolczynski-NOAA in #2482
• Revert file name changes in off-line post by @WenMeng-NOAA in #2499
• Add rocoto sh tag, script to check netcdf file and apply this to check ocean output by @aerorahul in #2484
• Fix paths for refactored prepocnobs task by @AndrewEichmann-NOAA in #2504
• Add C768 and C1152 S2SW test cases by @JessicaMeixner-NOAA in #2509
• CI Refactoring and STALLED case detection by @TerrenceMcGuinness-NOAA in #2488
• Enable using the FV3_global_nest_v1 CCPP suite by @guoqing-noaa in #2512
• Update gsi_enkf.sh hash to 457510c by @RussTreadon-NOAA in #2514
• Adding more cycles to gempak script gfs_meta_sa2.sh by @GwenChen-NOAA in #2518
• Update gdas_gsibec_ver to 20240416 by @RussTreadon-NOAA in #2497
• Update parm/transfer list files to match vetted GFSv16 set by @KateFriedman-NOAA in #2517
• Add restart on failure capability for the forecast executable by @aero...

This is a re-release of the prior GFSv16.3.16 version. This update corrects a missed prepobs version change for developer run mode. The operational GFS was unaffected since it does not directly reference the prepobs package in operations (the upstream obsproc system does). The prepobs package also updated in operations alongside the obsproc/v1.2 upgrade with prepobs now being v1.1.0.

What's Changed

• Update prepobs to v1.1.0 in GFSv16.3.16 by @KateFriedman-NOAA in #2849

Full Changelog: gfs.v16.3.16...gfs.v16.3.16.1

See the prior GFSv16.3.16 release for full details on that version.

RFC 12812 - add the GDAS WDQMS (WIGOS Data Quality Monitoring System) job

On WCOSS2, implement GFS v16.3.16. The GFS v16.3.16 update adds the GDAS WDQMS (WIGOS Data Quality Monitoring System) data
which is 6-hourly summaries of observation reports.

The objective of this new job is to:

• support the WMO Integrated Global Observing System (WIGOS) WDQMS project
• ensure that WMO observational data and products are reliable and correspond to agreed-upon needs
• 6-hourly summaries include surface (synop), upper-air, and surface marine (ship and buoy) observations

Was to be implemented on June 26, 1400Z to 1900Z; postponed due to College Park hardware issues; was implemented July 2, 1400Z to 2000Z.

See the SCN at: https://www.weather.gov/media/notification/pdf_2023_24/scn24-65_gfs_v16.3.16.pdf

What's Changed

• GFSv16.3.16 - add GDAS WDQMS (WIGOS Data Quality Monitoring System) job by @KateFriedman-NOAA in #2746

Full Changelog: gfs.v16.3.15...gfs.v16.3.16

Supported resolutions and platforms

WCOSS2 only

• C768C384L127 cycled & C768 forecast-only
• C384C192L127 cycled & C384 forecast-only
• C192C96L127 cycled & C192 forecast-only

Supported verification packages

• Fit2Obs v1.5
• METplus EMC_verif-global.v2.10.0

Setup steps

1. Quick clone and build instructions

git clone -b gfs.v16.3.16 https://github.com/NOAA-EMC/global-workflow.git
cd global-workflow/sorc
sh checkout.sh
sh build_all.sh
sh link_fv3gfs.sh emc wcoss2

Follow additional build and link instructions here: https://github.com/NOAA-EMC/global-workflow/wiki/Clone-and-build-global-workflow/08b2ec4d0956ea1936d567d9a1ac32614aedfec1

2. Run experiment setup scripts

See the following global-workflow wiki page: https://github.com/NOAA-EMC/global-workflow/wiki/Run-setup-scripts-to-generate-experiment/c85d5c2f87b298b6f0536a2fc2fb425a092c70fc

Note: the above wiki links are to older revisions of the global-workflow GitHub wiki (before updates were made for GFSv17/develop). Use wcoss2 for the machine where needed (e.g. link_fv3gfs.sh script) and ignore references to now unsupported machines (e.g. WCOSS-Dell and WCOSS-Cray).

Report any issues to @KateFriedman-NOAA

RFC 12604 - Upgrade to OBSPROC v1.2.0

On WCOSS2, implement the OBSPROC v1.2.0 upgrade along with downstream models' updates. This upgrade will:

• Expand the global observation datasets (gfs and gdas) with high vertical resolution radiosonde profiles, and high spatial and temporal resolution satellite winds tailored for hurricane events.
• Expand the rap observation dataset with high vertical resolution radiosonde profiles. It will also increase the volume of mesonet observations across all regional networks and expand the capacity of SDMEDIT list entries per observation type.
• Several internal BUFR-lib related arrays were increased to accommodate increasing obs data volumes.
• A number of obs data sets were added to NOMADS (see SCN).
• Turn off the obsproc_air_now job since the EPA AirNow data was stopped.

What's Changed

• Updates for obsproc/v1.2 in operations by @KateFriedman-NOAA in #2616

Full Changelog: gfs.v16.3.14...gfs.v16.3.15

Supported resolutions and platforms

• C768C384L127 cycled & C768 forecast-only on WCOSS2
• C384C192L127 cycled & C384 forecast-only on WCOSS2
• C192C96L127 cycled & C192 forecast-only on WCOSS2

Supported verification packages

• Fit2Obs v1.5
• METplus EMC_verif-global.v2.10.0

Setup steps

1. Quick clone and build instructions

git clone -b gfs.v16.3.15 https://github.com/NOAA-EMC/global-workflow.git
cd global-workflow/sorc
sh checkout.sh
sh build_all.sh
sh link_fv3gfs.sh emc wcoss2

Follow additional build and link instructions here: https://github.com/NOAA-EMC/global-workflow/wiki/Clone-and-build-global-workflow/08b2ec4d0956ea1936d567d9a1ac32614aedfec1

2. Run experiment setup scripts

See the following global-workflow wiki page: https://github.com/NOAA-EMC/global-workflow/wiki/Run-setup-scripts-to-generate-experiment/c85d5c2f87b298b6f0536a2fc2fb425a092c70fc

Note: the above wiki links are to older revisions of the global-workflow GitHub wiki (before updates were made for GFSv17/develop). Use wcoss2 for the machine where needed (e.g. link_fv3gfs.sh script) and ignore references to now unsupported machines (e.g. WCOSS-Dell and WCOSS-Cray).

Report any issues to @KateFriedman-NOAA

RFC 12411 - Remove ldebug=true flags from wave post jobs

On WCOSS2, remove the "-ldebug" flag from the gfs_wave_postnt, gfs_wave_post_bndpnt, and gfs_wave_post_bndpntbll jobs for the GFS WAVE model. After the recent Lustre upgrade on WCOSS2, the ldebug flag provided by GDIT is no longer needed since runtimes have stabilized for the GFS WAVE parallel.

Implemented on March 26 during the 12z cycle.

Supported resolutions and platforms:

• C768C384L127 cycled & C768 forecast-only on WCOSS2
• C384C192L127 cycled & C384 forecast-only on WCOSS2
• C192C96L127 cycled & C192 forecast-only on WCOSS2

Supported verification packages:

• Fit2Obs v1.5
• METplus EMC_verif-global.v2.10.0

Setup steps:

1. Quick clone and build instructions

git clone -b gfs.v16.3.14 https://github.com/NOAA-EMC/global-workflow.git
cd global-workflow/sorc
sh checkout.sh
sh build_all.sh
sh link_fv3gfs.sh emc wcoss2

Follow additional build and link instructions here: https://github.com/NOAA-EMC/global-workflow/wiki/Clone-and-build-global-workflow/08b2ec4d0956ea1936d567d9a1ac32614aedfec1

2. Run experiment setup scripts

See the following global-workflow wiki page: https://github.com/NOAA-EMC/global-workflow/wiki/Run-setup-scripts-to-generate-experiment/c85d5c2f87b298b6f0536a2fc2fb425a092c70fc

Note: the above wiki links are to older revisions of the global-workflow GitHub wiki (before updates were made for GFSv17/develop). Use wcoss2 for the machine where needed (e.g. link_fv3gfs.sh script) and ignore references to now unsupported machines (e.g. WCOSS-Dell and WCOSS-Cray).

Report any issues to @KateFriedman-NOAA

RFC 12134 - WAFS high resolution upgrade

On WCOSS2, upgrade the GFS to v16.3.13. With this update, NCEP, in coordination with UKMO (UK Met Office), is updating the GFS to produce high resolution WAFS output and stop producing blended 1.25 deg WAFS files in order to meet a 2023 ICAO (International Civil Aviation Organization) milestone.

Implemented on January 24 during the 12z cycle.

Supported resolutions and platforms:

• C768C384L127 cycled & C768 forecast-only on WCOSS2
• C384C192L127 cycled & C384 forecast-only on WCOSS2
• C192C96L127 cycled & C192 forecast-only on WCOSS2

Supported verification packages:

• Fit2Obs v1.5
• METplus EMC_verif-global.v2.10.0

Setup steps:

1. Quick clone and build instructions

git clone -b gfs.v16.3.13 https://github.com/NOAA-EMC/global-workflow.git
cd global-workflow/sorc
sh checkout.sh
sh build_all.sh
sh link_fv3gfs.sh emc wcoss2

Follow additional build and link instructions here: https://github.com/NOAA-EMC/global-workflow/wiki/Clone-and-build-global-workflow/08b2ec4d0956ea1936d567d9a1ac32614aedfec1

2. Run experiment setup scripts

See the following global-workflow wiki page: https://github.com/NOAA-EMC/global-workflow/wiki/Run-setup-scripts-to-generate-experiment/c85d5c2f87b298b6f0536a2fc2fb425a092c70fc

Note: the above wiki links are to older revisions of the global-workflow GitHub wiki (before updates were made for GFSv17/develop). Use wcoss2 for the machine where needed (e.g. link_fv3gfs.sh script) and ignore references to now unsupported machines (e.g. WCOSS-Dell and WCOSS-Cray).

Report any issues to @KateFriedman-NOAA

RFC 12011 - Turn on active assimilation of NOAA-21 radiances

On WCOSS2, fix files global_satinfo.txt and global_convinfo.txt are updated to begin actively assimilating NOAA-21 radiances into the GFS. A new GSI tag (gfsda.v16.3.12) is included.

Implemented December 18 2023 during 12z cycle.

Supported resolutions and platforms:

• C768C384L127 cycled & C768 forecast-only on WCOSS2
• C384C192L127 cycled & C384 forecast-only on WCOSS2
• C192C96L127 cycled & C192 forecast-only on WCOSS2

Supported verification packages:

• Fit2Obs v1.5
• METplus EMC_verif-global.v2.10.0

Setup steps:

1. Quick clone and build instructions

git clone -b gfs.v16.3.12 https://github.com/NOAA-EMC/global-workflow.git
cd global-workflow/sorc
sh checkout.sh
sh build_all.sh
sh link_fv3gfs.sh emc wcoss2

Follow additional build and link instructions here: https://github.com/NOAA-EMC/global-workflow/wiki/Clone-and-build-global-workflow/08b2ec4d0956ea1936d567d9a1ac32614aedfec1

2. Run experiment setup scripts

See the following global-workflow wiki page: https://github.com/NOAA-EMC/global-workflow/wiki/Run-setup-scripts-to-generate-experiment/c85d5c2f87b298b6f0536a2fc2fb425a092c70fc

Note: the above wiki links are to older revisions of the global-workflow GitHub wiki (before updates were made for GFSv17/develop). Use wcoss2 for the machine where needed (e.g. link_fv3gfs.sh script) and ignore references to now unsupported machines (e.g. WCOSS-Dell and WCOSS-Cray).

Report any issues to @KateFriedman-NOAA