Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NCEP regtest module updates: uses spack-stack/1.5.0, includes scotch/7.0.4 #1137

Merged

Conversation

MatthewMasarik-NOAA
Copy link
Collaborator

@MatthewMasarik-NOAA MatthewMasarik-NOAA commented Dec 6, 2023

Pull Request Summary

Updates the module environment used for NCEP's regtests on RDHPCS machines to use the spack-stack/1.5.0 stack which includes scotch/7.0.4.

Description

  • Key points about this update:
    • Switches to spack -based spack-stack module stack. Previously the hpc-stack module stack has been used.
    • A module version of scotch is used. Previously the export-ed SCOTCH_PATH parameter pointed to a local installation.
    • The RDHPCS machines this update applies to are hera and orion.
  • Mention any labels that should be added:
    • enhancement
  • Are answer changes expected from this PR?
    • Yes.
  • Please describe the changes and the reason why in addition to which of the following labels would apply:
    • Some answers changes are expected and due only to the change in the module environment.
    • out_grd change, out_pnt change, restart file change, Regression test

Issue(s) addressed

Commit Message

NCEP regtest module updates: uses spack-stack/1.5.0, includes scotch/7.0.4

Check list

Testing

  • How were these changes tested?
    • Answers cannot be expected to reproduce between differing modules stack environments, so the PR branch was run against itself on both hera and orion.
  • Are the changes covered by regression tests? (If not, why? Do new tests need to be added?)
    • NA (we cannot/do not test for module environment changes).
  • Have the matrix regression tests been run (if yes, please note HPC and compiler)?
    • hera/intel, orion/intel.
  • Please indicate the expected changes in the regression test output, (Note the list of known non-identical tests.)
    • Changes are expected from previous stack. The PR branch runs against itself have no unexpected changes.
  • Please provide the summary output of matrix.comp (matrix.Diff.txt, matrixCompFull.txt and matrixCompSummary.txt):

orion
Note: The (0 files differ) all checkout as soft links from input/ that have gone bad. It is completely benign. Once these are removed only the known non-b4b are left.

**********************************************************************
********************* non-identical cases ****************************
**********************************************************************
mww3_test_02/./work_PR3_UQ_MPI_b_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_d                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_d                     (0 files differ)
mww3_test_02/./work_PR2_UNO_MPI_a                     (0 files differ)
mww3_test_02/./work_PR2_UQ_a                     (0 files differ)
mww3_test_02/./work_PR2_UNO_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_b                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_a                     (0 files differ)
mww3_test_02/./work_PR1_a                     (0 files differ)
mww3_test_02/./work_PR1_MPI_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_c_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_d_c                     (0 files differ)
mww3_test_02/./work_PR2_UNO_d                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_c_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_c_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_d_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_b_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_a_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_a                     (0 files differ)
mww3_test_02/./work_PR3_UNO_b_c                     (0 files differ)
mww3_test_02/./work_PR2_UNO_b                     (0 files differ)
mww3_test_02/./work_PR2_UQ_MPI_a                     (0 files differ)
mww3_test_02/./work_PR2_UQ_MPI_c                     (0 files differ)
mww3_test_02/./work_PR2_UQ_b                     (0 files differ)
mww3_test_02/./work_PR2_UNO_MPI_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_a_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_d                     (0 files differ)
mww3_test_02/./work_PR1_MPI_a                     (0 files differ)
mww3_test_02/./work_PR3_UQ_b                     (0 files differ)
mww3_test_02/./work_PR1_MPI_d                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_b_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_b                     (0 files differ)
mww3_test_02/./work_PR1_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_d_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_d                     (0 files differ)
mww3_test_02/./work_PR2_UQ_MPI_b                     (0 files differ)
mww3_test_02/./work_PR2_UNO_MPI_d                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_a                     (0 files differ)
mww3_test_02/./work_PR2_UQ_c                     (0 files differ)
mww3_test_02/./work_PR1_b                     (0 files differ)
mww3_test_02/./work_PR2_UNO_a                     (0 files differ)
mww3_test_02/./work_PR3_UNO_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_b                     (0 files differ)
mww3_test_02/./work_PR2_UNO_MPI_b                     (0 files differ)
mww3_test_02/./work_PR3_UQ_a                     (0 files differ)
mww3_test_02/./work_PR2_UQ_MPI_d                     (0 files differ)
mww3_test_02/./work_PR2_UQ_d                     (0 files differ)
mww3_test_02/./work_PR1_d                     (0 files differ)
mww3_test_02/./work_PR3_UNO_a_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_a_c                     (0 files differ)
mww3_test_02/./work_PR1_MPI_b                     (0 files differ)
mww3_test_02/./work_PR3_UNO_c_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_d_c                     (0 files differ)
mww3_test_03/./work_PR1_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UNO_MPI_d2                     (16 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2                     (14 files differ)
mww3_test_03/./work_PR3_UNO_MPI_e                     (1 files differ)
mww3_test_03/./work_PR3_UQ_MPI_e                     (1 files differ)
mww3_test_03/./work_PR1_MPI_d2                     (9 files differ)
mww3_test_03/./work_PR2_UQ_MPI_d2                     (11 files differ)
mww3_test_03/./work_PR3_UQ_MPI_e_c                     (1 files differ)
mww3_test_03/./work_PR3_UNO_MPI_e_c                     (1 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2                     (12 files differ)
mww3_test_03/./work_PR2_UNO_MPI_e                     (1 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2_c                     (15 files differ)
mww3_test_03/./work_PR2_UQ_MPI_e                     (1 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2_c                     (18 files differ)
mww3_test_09/./work_MPI_ASCII                     (0 files differ)
ww3_ta1/./work_UPD5_U                     (0 files differ)
ww3_ta1/./work_UPD6_U                     (0 files differ)
ww3_ta1/./work_UPD0F_O                     (0 files differ)
ww3_ta1/./work_UPD5_O                     (0 files differ)
ww3_ta1/./work_UPD2_O                     (0 files differ)
ww3_ta1/./work_UPD6_O                     (0 files differ)
ww3_ta1/./work_UPD5_U_cap                     (0 files differ)
ww3_ta1/./work_UPD6_U_cap                     (0 files differ)
ww3_ta1/./work_UPD3_O                     (0 files differ)
ww3_ta1/./work_UPD3_U                     (0 files differ)
ww3_ta1/./work_UPD2_U                     (0 files differ)
ww3_ta1/./work_UPD2_U_cap                     (0 files differ)
ww3_ta1/./work_UPD3_U_cap                     (0 files differ)
ww3_tp2.10/./work_MPI_OMPH                     (7 files differ)
ww3_tp2.16/./work_MPI_OMPH                     (4 files differ)
ww3_tp2.6/./work_ST4_ASCII                     (0 files differ)
ww3_ufs1.3/./work_a                     (3 files differ)
 
**********************************************************************
************************ identical cases *****************************
**********************************************************************

orion.matrixCompSummary.txt
orion.matrixDiff.txt
orion.matrixCompFull.txt

hera
Here only known non-b4bs present.

**********************************************************************
********************* non-identical cases ****************************
**********************************************************************
mww3_test_03/./work_PR1_MPI_e                     (1 files differ)
mww3_test_03/./work_PR3_UQ_MPI_e_c                     (1 files differ)
mww3_test_03/./work_PR3_UNO_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UQ_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UNO_MPI_d2                     (12 files differ)
mww3_test_03/./work_PR1_MPI_d2                     (8 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2_c                     (8 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2_c                     (16 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2                     (17 files differ)
mww3_test_03/./work_PR2_UQ_MPI_d2                     (16 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2                     (15 files differ)
mww3_test_09/./work_MPI_ASCII                     (0 files differ)
ww3_tp2.10/./work_MPI_OMPH                     (7 files differ)
ww3_tp2.16/./work_MPI_OMPH                     (4 files differ)
ww3_tp2.6/./work_ST4_ASCII                     (0 files differ)
ww3_ufs1.3/./work_a                     (3 files differ)
 
**********************************************************************
************************ identical cases *****************************
**********************************************************************

hera.matrixCompSummary.txt
hera.matrixDiff.txt
hera.matrixCompFull.txt

@MatthewMasarik-NOAA MatthewMasarik-NOAA marked this pull request as ready for review December 6, 2023 21:37
@MatthewMasarik-NOAA
Copy link
Collaborator Author

@JessicaMeixner-NOAA PR header will be cleaned up tomorrow.

@MatthewMasarik-NOAA
Copy link
Collaborator Author

@JessicaMeixner-NOAA PR header is cleaned up and ready.

Copy link
Collaborator

@JessicaMeixner-NOAA JessicaMeixner-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making this update Matt. It's good that things are now running on orion again.

Testing orion vs itself I got the following diffs:

**********************************************************************
********************* non-identical cases ****************************
**********************************************************************
mww3_test_02/./work_PR1_MPI_c                     (1 files differ)
mww3_test_03/./work_PR2_UNO_MPI_d2                     (8 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2                     (17 files differ)
mww3_test_03/./work_PR3_UNO_MPI_e                     (1 files differ)
mww3_test_03/./work_PR3_UQ_MPI_e                     (1 files differ)
mww3_test_03/./work_PR1_MPI_d2                     (10 files differ)
mww3_test_03/./work_PR2_UQ_MPI_d2                     (16 files differ)
mww3_test_03/./work_PR3_UQ_MPI_e_c                     (1 files differ)
mww3_test_03/./work_PR3_UNO_MPI_e_c                     (1 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2                     (17 files differ)
mww3_test_03/./work_PR2_UNO_MPI_e                     (1 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2_c                     (16 files differ)
mww3_test_03/./work_PR2_UQ_MPI_e                     (1 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2_c                     (16 files differ)
mww3_test_09/./work_MPI_ASCII                     (0 files differ)
ww3_tp2.10/./work_MPI_OMPH                     (7 files differ)
ww3_tp2.16/./work_MPI_OMPH                     (4 files differ)
ww3_tp2.6/./work_ST4_ASCII                     (0 files differ)
ww3_ufs1.3/./work_a                     (3 files differ)

matrixCompFull.txt
matrixCompSummary.txt
matrixDiff.txt

So essentially the normal + an odd log file diff for mww3_test_02/./work_PR1_MPI_c which is not concerning.
For record keeping here's the diffs vs develop (which did not have an existing scotch library so not all tests succeeded):

matrixCompFull.txt
matrixCompSummary.txt

For Hera, which is what most regtests are looking for we have the following diffs when comparing against itself:

**********************************************************************
********************* non-identical cases ****************************
**********************************************************************
mww3_test_03/./work_PR2_UNO_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UNO_MPI_d2                     (12 files differ)
mww3_test_03/./work_PR1_MPI_d2                     (9 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2_c                     (12 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2_c                     (16 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2                     (18 files differ)
mww3_test_03/./work_PR2_UQ_MPI_d2                     (16 files differ)
mww3_test_03/./work_PR3_UQ_MPI_e                     (1 files differ)
mww3_test_03/./work_PR3_UNO_MPI_e_c                     (1 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2                     (16 files differ)
mww3_test_09/./work_MPI_ASCII                     (0 files differ)
ww3_tp2.10/./work_MPI_OMPH                     (7 files differ)
ww3_tp2.16/./work_MPI_OMPH                     (4 files differ)
ww3_tp2.6/./work_ST4_ASCII                     (0 files differ)
ww3_ufs1.3/./work_a                     (3 files differ)

matrixCompFull.txt
matrixCompSummary.txt
matrixDiff.txt

For tracking purposes here are the differences with develop:

matrixCompFull.txt
matrixCompSummary.txt

The main differences here are that the gribs do not match themselves and scotch tests have differences that are small in magnitude.

@JessicaMeixner-NOAA JessicaMeixner-NOAA merged commit ff1b9e9 into NOAA-EMC:develop Dec 7, 2023
20 checks passed
@MatthewMasarik-NOAA
Copy link
Collaborator Author

@JessicaMeixner-NOAA thanks for the detailed review. It's great to have that info for posterity.

@MatthewMasarik-NOAA MatthewMasarik-NOAA deleted the regtests/spack-stack branch December 20, 2023 22:57
miguelsolanocordoba added a commit to wavespotter/WW3 that referenced this pull request Apr 19, 2024
* Bugfix - initialised VD and VS to zero in w3srcemd. (NOAA-EMC#1037)

* More efficient test for binary files in matrix.comp (NOAA-EMC#1035)

* Tidy up of pre-processor directives and unused variables in w3srcemd.F90 (NOAA-EMC#1010)

* Correct typo in w3srcemd.F90 pre-processor directive. (NOAA-EMC#1039)

* minor bugfix for matrix grepping on keywords (NOAA-EMC#1049)

* Stop masking group 1 output where icec > icen (NOAA-EMC#1019)

* Doxygen documentation added, 8th subset.(NOAA-EMC#1046)

* NC4 ,F90 ,XX0 switches removed from ww3_tp2.19 regtest (NOAA-EMC#1054)

* CI:  Fix for Intel scripts. GNU scripts updated. (NOAA-EMC#1064)

* correct the computation of QP parameter, add QKK output parameter, change UST scale factor (NOAA-EMC#1050)

* correct issue with ww3_multi when requesting restart2 and using nml file instead of inp file (NOAA-EMC#1070)

* correct calendar for track netcdf output (NOAA-EMC#1079)

* Fix missing mod_def.ww3 file in multigrid regression tests for track output (NOAA-EMC#1091)

* STAB3: fix cmake build for ST4 or ST3 (NOAA-EMC#1086)

* new feature to output out_grd.ww3, out_pnt.ww3 and mod_def.ww3 both in binary and ascii format using switch ASCII. (NOAA-EMC#1089)

* Update local unit number arrays (NDS, MDS) to be same size of array defined in w3odatmd (size=15). Also, defined unit numbers for NDS(14) and NDS(15). (NOAA-EMC#1098)

* Removed code referencing PHIOC in output section for PHICE in ww3_ounf (NOAA-EMC#1093)

* implementation of the GQM (Gaussian Quadrature Method) to replace the DIA in NL1 or NL2. (NOAA-EMC#1083)

* update logic to ensure you are not accessing uninitialized dates (NOAA-EMC#1114)

* Initialised S and D arrays in W3SDB1 before potential early return if zero energy. (NOAA-EMC#1115)

* ww3_ounp.F90:  x/y units attribute corrected from 'm' to 'km' (NOAA-EMC#1088)

* Bugfix: Assign unit numbers to ASCII gridded/point output in multi-grid mode. (NOAA-EMC#1118)

* correct bugs to run correctly GQM implementation (NOAA-EMC#1127)

* Adding documentation to w3iopo() in preparation for code for NOAA-EMC#682. (NOAA-EMC#1131)

* NCEP regtest module updates: uses spack-stack/1.5.0, includes scotch/7.0.4 (NOAA-EMC#1137)

* Minor update to ncep regtests (NOAA-EMC#1138)

* Updated intel workflow to install oneapi compilers from new location. (NOAA-EMC#1157)

* Add unit test for points I/O code. (NOAA-EMC#1158)

* Update Intel CI (relocate /usr/local; ensure intel-oneapi-mpi; use ubuntu-latest) (NOAA-EMC#1161)

* remove lookup table for ST4 to speed up computation and clean up the ST4 code (NOAA-EMC#1124)

Co-authored-by: Fabrice Ardhuin <[email protected]>

* initialize USSP_WN for mod_def (NOAA-EMC#1165)

* Introduce IC4M8 and IC4M9 to WW3 (NOAA-EMC#1176)

* clean up and add ST4 variables (NOAA-EMC#1181)

* w3fld1md.F90: fix divide by zero in CRIT2 parameter (NOAA-EMC#1184)

* ww3_prnc.F90: fix out-of-scope grid index write statement (NOAA-EMC#1185)

* Bugfix: address potential divide-by-zero in APPENDTAIL (NOAA-EMC#1188)

Co-authored-by: Denise Worthen <[email protected]>

* Provide initial drying of cells with depth < ZLIM for SMC grid. (NOAA-EMC#1192)

* Output OMP threading info to screen when running ww3_shel/ww3_multi compiled with the OMPG switch. Also fixes truncation of build.log when running run_cmake_build. (NOAA-EMC#1191)

* Added screen output showing number of threads when OMP enabled.

* update build to get more info in logs (NOAA-EMC#46)

---------

Co-authored-by: Jessica Meixner <[email protected]>

* update run_cmake_test to catch build errors and exit (NOAA-EMC#1194)

* fix merge conflicts

* Fix gustiness bug, as suggst by Pieter

* Change USTARsigma to WAM implementation

---------

Co-authored-by: Chris Bunney <[email protected]>
Co-authored-by: Mickael Accensi <[email protected]>
Co-authored-by: Benoit Pouliot <[email protected]>
Co-authored-by: Matthew Masarik <[email protected]>
Co-authored-by: Ghazal-Mohammadpour <[email protected]>
Co-authored-by: Jessica Meixner <[email protected]>
Co-authored-by: Biao Zhao <[email protected]>
Co-authored-by: Edward Hartnett <[email protected]>
Co-authored-by: Alex Richert <[email protected]>
Co-authored-by: Fabrice Ardhuin <[email protected]>
Co-authored-by: W. Erick Rogers <[email protected]>
Co-authored-by: Denise Worthen <[email protected]>
Co-authored-by: Camille Teicheira <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update RDHPCS regtest modules for spack-stack 1.5.0 and scotch/7.0.4
2 participants