Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/st4table #1124

Merged
merged 24 commits into from
Jan 16, 2024
Merged

Conversation

mickaelaccensi
Copy link
Collaborator

@mickaelaccensi mickaelaccensi commented Nov 10, 2023

Pull Request Summary

remove lookup table for ST4 and clean up the ST4 code

Description

remove lookup table in ST4 to speed up computation
clean up ST4 code
correct uninitialized variables

Issue(s) addressed

Commit Message

remove lookup table for ST4 to speed up computation and clean up the ST4 code

Check list

Testing

  • How were these changes tested? matrix
  • Are the changes covered by regression tests? (If not, why? Do new tests need to be added?) ww3_ts1
  • Have the matrix regression tests been run (if yes, please note HPC and compiler)? yes
  • Please indicate the expected changes in the regression test output, (Note the list of known non-identical tests.) mod_def.ww3, out_grd.ww3, out_pnt.ww3, restart.ww3 for all runs using ST4 switch
  • Please provide the summary output of matrix.comp (matrix.Diff.txt, matrixCompFull.txt and matrixCompSummary.txt)
    matrixCompFull.txt
    matrixCompSummary.txt
    matrixDiff.txt is too big to be uploaded

expected changes :
ww3_ts1 : output parameters slight changes (work_T7**)
ww3_ts2 : only mod_def since last bugfix
ww3_ts3 : only mod_def since last bugfix
ww3_ts4 : only mod_def
ww3_tp2.6 : only mod_def
ww3_tp2.6/work_pdlib : not b4b due to scotch library on Ifremer HPC (also on develop branch)
ww3_tp2.18 : only mod_def
ww3_tp2.14 : only mod_def since last bugfix
ww3_tic2.3 : only mod_def
ww3_tic1.4 : only mod_def
ww3_ta1 : only mod_def since last bugfix
mww3_test_08 : only mod_def since last bugfix
mww3_test_05 : only mod_def since last bugfix
mww3_test_03 : known not b4b
ww3_tp1.8 : output parameters changes for taw, two and foc (partially correct issue in T570)

could be expected differences :
ww3_tp2.21 : output parameters slight changes (work_b and work_mb)
ww3_tp2.17 : output parameters slight changes (work_c, work_mc, work_mc1)
ww3_tp2.15 : output parameters slight changes

@MatthewMasarik-NOAA
Copy link
Collaborator

@mickaelaccensi, this PR is up next, I'll put the regression tests in now!

@MatthewMasarik-NOAA
Copy link
Collaborator

Hi @mickaelaccensi, the header mentions the matrix.comp tests were running when this was submitted. Have those completed?

@mickaelaccensi
Copy link
Collaborator Author

completed but I'm not really happy with the differences... they are many regtests with different results so I have to discuss with Fabrice first. I'll keep informed asap

@MatthewMasarik-NOAA
Copy link
Collaborator

Hi @mickaelaccensi, okay, thank you for the update. Yes, please keep us informed when you find out more.

@MatthewMasarik-NOAA MatthewMasarik-NOAA mentioned this pull request Nov 27, 2023
4 tasks
@MatthewMasarik-NOAA
Copy link
Collaborator

@mickaelaccensi, with #1127 merged I'm going to start in on this PR. Do you have the matrix logs? This is a big PR with a number of expected differences, though I'm going to try to keep it moving along as Chris needs it merged for his work.

@mickaelaccensi
Copy link
Collaborator Author

@MatthewMasarik-NOAA I first have to solve the issues found in regtests mww3_test_05, ww3_ts2 and ww3_ts3 which contains too large Hs values. I'll work on it this week and keep you informed

@MatthewMasarik-NOAA
Copy link
Collaborator

Hi @mickaelaccensi, okay I understand, thanks for the update. I'll wait to hear from you to proceed.

@mickaelaccensi
Copy link
Collaborator Author

finally ! the branch is ready to be merged !
sorry for this long delay @ukmo-ccbunney

@JessicaMeixner-NOAA
Copy link
Collaborator

@mickaelaccensi - I'll be doing the testing for this PR. Our main machine we test on is down for maintenance today, but I will submit tests first thing tomorrow.

@MatthewMasarik-NOAA
Copy link
Collaborator

Thanks for the updates @mickaelaccensi! I will be going on leave so @JessicaMeixner-NOAA is kindly taking care of this review.

@JessicaMeixner-NOAA
Copy link
Collaborator

@mickaelaccensi my list of not b4b tests is mostly the same as yours with a quick glance. I have a few more diffs in the ufs tests that we run and you do not and so I want to look at that. Also, since there's quite a few differences, I'm going to do a sanity test that we can reproduce answers with this PR. Our queues are pretty slow right now and despite submitting the runs last night, they have not yet started running, so this is unfortunately going to take longer than I'd hope. I'll keep you posted if I run into anything unexpected or have questions.

@@ -294,17 +293,18 @@ SUBROUTINE W3SPR4 (A, CG, WN, EMEAN, FMEAN, FMEAN1, WNMEAN, &
DO IK=1, NK
EB(IK) = 0.
EB2(IK) = 0.
SIGFAC=SIG(IK)**SSDSC(12) * DDEN(IK) / CG(IK)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's never a scenario where CG(IK) =0 is there? (I'm assuming no, but if so we should have some sort of catch so we do not divide by zero).

@JessicaMeixner-NOAA
Copy link
Collaborator

@mickaelaccensi the queue times are really long right now, so unfortunately I'm still in the queue waiting. In the meantime, I went back through the regtests and things look as expected:

**********************************************************************
********************* non-identical cases ****************************
**********************************************************************
mww3_test_03/./work_PR3_UNO_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UQ_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UNO_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UNO_MPI_d2                     (12 files differ)
mww3_test_03/./work_PR1_MPI_d2                     (10 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2_c                     (16 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2_c                     (16 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2                     (12 files differ)
mww3_test_03/./work_PR2_UQ_MPI_d2                     (16 files differ)
mww3_test_03/./work_PR3_UNO_MPI_e_c                     (1 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2                     (15 files differ)
mww3_test_05/./work_ST4_PR3_UQ_MPI_OMPH                     (10 files differ)
mww3_test_05/./work_ST4_PR3_UNO_MPI                     (10 files differ)
mww3_test_05/./work_ST4_PR3_UQ_MPI                     (10 files differ)
mww3_test_05/./work_ST4_PR2_UQ_MPI                     (10 files differ)
mww3_test_05/./work_ST4_PR2_UNO_OMP                     (10 files differ)
mww3_test_05/./work_ST4_PR3_UNO_OMP                     (10 files differ)
mww3_test_05/./work_ST4_PR3_UNO_MPI_OMPH                     (10 files differ)
mww3_test_05/./work_ST4_PR1_MPI                     (10 files differ)
mww3_test_05/./work_ST4_PR1_MPI_OMPH                     (10 files differ)
mww3_test_05/./work_ST4_PR1_OMP                     (10 files differ)
mww3_test_05/./work_ST4_PR3_UQ_OMP                     (10 files differ)
mww3_test_05/./work_ST4_PR2_UNO_MPI                     (10 files differ)
mww3_test_05/./work_ST4_PR2_UNO_MPI_OMPH                     (10 files differ)
mww3_test_05/./work_ST4_PR2_UQ_OMP                     (10 files differ)
mww3_test_05/./work_ST4_PR2_UQ_MPI_OMPH                     (10 files differ)
mww3_test_08/./work_ST4_PR3_UQ_MPI                     (4 files differ)
mww3_test_08/./work_lowres                     (2 files differ)
mww3_test_09/./work_MPI_ASCII                     (0 files differ)
ww3_ta1/./work_UPD0F_O                     (2 files differ)
ww3_ta1/./work_UPD0F_U                     (2 files differ)
ww3_ta1/./work_UPD2_U_cap                     (2 files differ)
ww3_ta1/./work_UPD3_U                     (2 files differ)
ww3_ta1/./work_UPD5_U_cap                     (2 files differ)
ww3_ta1/./work_UPD6_O                     (2 files differ)
ww3_ta1/./work_UPD2_U                     (2 files differ)
ww3_ta1/./work_UPD5_O                     (2 files differ)
ww3_ta1/./work_UPD5_U                     (2 files differ)
ww3_ta1/./work_UPD3_U_cap                     (2 files differ)
ww3_ta1/./work_UPD3_O                     (2 files differ)
ww3_ta1/./work_UPD6_U                     (2 files differ)
ww3_ta1/./work_UPD2_O                     (2 files differ)
ww3_ta1/./work_UPD6_U_cap                     (2 files differ)
ww3_tic1.4/./work_IC1IS2_1000                     (1 files differ)
ww3_tic1.4/./work_IC2IS2_IC2d                     (1 files differ)
ww3_tic1.4/./work_IC2IS2_IC2b                     (1 files differ)
ww3_tic1.4/./work_IC0IS2_1000                     (1 files differ)
ww3_tic2.3/./work_IC2IS2creep                     (1 files differ)
ww3_tic2.3/./work_IC2IS2scat                     (1 files differ)
ww3_tic2.3/./work_IC2IS2dissip                     (1 files differ)
ww3_tp1.8/./work_PR3_UQ                     (6 files differ)
ww3_tp2.10/./work_MPI                     (2 files differ)
ww3_tp2.10/./work_MPI_OMPH                     (9 files differ)
ww3_tp2.14/./work_OASACM6                     (2 files differ)
ww3_tp2.14/./work_OASACM3                     (2 files differ)
ww3_tp2.14/./work_OASACM5                     (2 files differ)
ww3_tp2.14/./work_OASACM4                     (2 files differ)
ww3_tp2.14/./work_OASACM                     (2 files differ)
ww3_tp2.14/./work_OASICM                     (2 files differ)
ww3_tp2.14/./work_OASACM2                     (2 files differ)
ww3_tp2.14/./work_OASOCM                     (2 files differ)
ww3_tp2.15/./work_PR3_UQ_RHO                     (7 files differ)
ww3_tp2.15/./work_5km                     (2 files differ)
ww3_tp2.15/./work_PR3_UQ_RHO_MPI                     (7 files differ)
ww3_tp2.15/./work_PR3_UQ                     (2 files differ)
ww3_tp2.15/./work_MPI_5km                     (2 files differ)
ww3_tp2.15/./work_PR3_UQ_MPI                     (2 files differ)
ww3_tp2.15/./work_ST4FLX5                     (7 files differ)
ww3_tp2.16/./work_MPI_OMPH                     (4 files differ)
ww3_tp2.17/./work_ma                     (2 files differ)
ww3_tp2.17/./work_a                     (2 files differ)
ww3_tp2.17/./work_mc1                     (2 files differ)
ww3_tp2.17/./work_mb                     (2 files differ)
ww3_tp2.17/./work_mc                     (2 files differ)
ww3_tp2.17/./work_ma1                     (2 files differ)
ww3_tp2.17/./work_c                     (2 files differ)
ww3_tp2.17/./work_b                     (2 files differ)
ww3_tp2.18/./work_TIDE_MPI                     (2 files differ)
ww3_tp2.18/./work_TIDE                     (2 files differ)
ww3_tp2.21/./work_ma                     (4 files differ)
ww3_tp2.21/./work_b_metis                     (2 files differ)
ww3_tp2.21/./work_a                     (2 files differ)
ww3_tp2.21/./work_mb                     (4 files differ)
ww3_tp2.21/./work_b                     (2 files differ)
ww3_tp2.6/./work_ST4                     (2 files differ)
ww3_tp2.6/./work_ST4_ASCII                     (3 files differ)
ww3_ts1/./work_Romero                     (directory not found)
ww3_ts1/./work_T701                     (directory not found)
ww3_ts1/./work_ST4_WRT                     (2 files differ)
ww3_ts1/./work_ST4_GMD                     (2 files differ)
ww3_ts1/./work_ST4                     (2 files differ)
ww3_ts1/./work_ST4_TSA                     (2 files differ)
ww3_ts1/./work_ST4_T700                     (6 files differ)
ww3_ts1/./work_T702                     (directory not found)
ww3_ts1/./work_ST4_T500                     (directory not found)
ww3_ts1/./work_T707GQM                     (6 files differ)
ww3_ts1/./work_T713GQM                     (6 files differ)
ww3_ts2/./work_ST4_PR1                     (2 files differ)
ww3_ts2/./work_ST4_PR3_UNO                     (2 files differ)
ww3_ts2/./work_ST4_PR3_UQ                     (2 files differ)
ww3_ts2/./work_ST4_PR2_UQ                     (2 files differ)
ww3_ts2/./work_ST4_PR2_UNO                     (2 files differ)
ww3_ts3/./work_ST4_PR3_UQ_MPI_OMPH                     (2 files differ)
ww3_ts3/./work_ST4_PR3_UNO_MPI                     (2 files differ)
ww3_ts3/./work_ST4_PR3_UQ_MPI                     (2 files differ)
ww3_ts3/./work_ST4_FLD2_MPI                     (2 files differ)
ww3_ts3/./work_ST4_PR2_UQ_MPI                     (2 files differ)
ww3_ts3/./work_ST4_PR2_UNO_OMP                     (2 files differ)
ww3_ts3/./work_ST4_PR3_UNO_OMP                     (2 files differ)
ww3_ts3/./work_ST4_PR3_UNO_MPI_OMPH                     (2 files differ)
ww3_ts3/./work_ST4_PR1_MPI                     (2 files differ)
ww3_ts3/./work_ST4_PR1_MPI_OMPH                     (2 files differ)
ww3_ts3/./work_ST4_FLD1_MPI                     (2 files differ)
ww3_ts3/./work_ST4_PR1_OMP                     (2 files differ)
ww3_ts3/./work_ST4_PR3_UQ_OMP                     (2 files differ)
ww3_ts3/./work_ST4_PR2_UNO_MPI                     (2 files differ)
ww3_ts3/./work_ST4_PR2_UNO_MPI_OMPH                     (2 files differ)
ww3_ts3/./work_ST4_PR2_UQ_OMP                     (2 files differ)
ww3_ts3/./work_ST4_PR2_UQ_MPI_OMPH                     (2 files differ)
ww3_ts4/./work_rg_shel_MPI                     (1 files differ)
ww3_ts4/./work_ug_MPI                     (1 files differ)
ww3_ts4/./work_rg_multi_MPI                     (3 files differ)
ww3_ufs1.1/./work_unstr_b                     (2 files differ)
ww3_ufs1.1/./work_c_npl                     (4 files differ)
ww3_ufs1.1/./work_unstr_a                     (2 files differ)
ww3_ufs1.1/./work_unstr_c                     (2 files differ)
ww3_ufs1.1/./work_d                     (4 files differ)
ww3_ufs1.1/./work_c_nth                     (4 files differ)
ww3_ufs1.1/./work_c                     (4 files differ)
ww3_ufs1.2/./work_a                     (12 files differ)
ww3_ufs1.2/./work_l                     (8 files differ)
ww3_ufs1.2/./work_c                     (14 files differ)
ww3_ufs1.2/./work_b                     (12 files differ)
ww3_ufs1.3/./work_a                     (13 files differ)

matrixCompFull.txt
matrixCompSummary.txt

I also did another check of the code and noted 2 places where we may or may not want to have a catch for divide by zero. In the meantime, I do want to make sure we can reproduce results before merging so apologies for the delay on that. Monday is a holiday here in the US, so it'll likely be 1/16 for merge right now. One other comment: Did we need to add any namelist description updates to model/inp/ww3_grid.inp?

@JessicaMeixner-NOAA
Copy link
Collaborator

@mickaelaccensi thank you for your patience with the slow queues on this end. I have finally been able to confirm everything is b4b with itself:

**********************************************************************
********************* non-identical cases ****************************
**********************************************************************
mww3_test_03/./work_PR1_MPI_e                     (1 files differ)
mww3_test_03/./work_PR3_UNO_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UQ_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UNO_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UNO_MPI_d2                     (16 files differ)
mww3_test_03/./work_PR1_MPI_d2                     (11 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2_c                     (10 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2_c                     (16 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2                     (18 files differ)
mww3_test_03/./work_PR2_UQ_MPI_d2                     (16 files differ)
mww3_test_03/./work_PR3_UNO_MPI_e_c                     (1 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2                     (16 files differ)
mww3_test_09/./work_MPI_ASCII                     (0 files differ)
ww3_tp2.10/./work_MPI_OMPH                     (6 files differ)
ww3_tp2.16/./work_MPI_OMPH                     (4 files differ)
ww3_tp2.6/./work_ST4_ASCII                     (0 files differ)
ww3_ufs1.3/./work_a                     (3 files differ)

Copy link
Collaborator

@JessicaMeixner-NOAA JessicaMeixner-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mickaelaccensi I'm approving this PR, however, it would be great if you could follow up on two things still:

  • Confirm the two spots in w3src4md will not ever be divide by zero (a follow-up PR for work around is sufficient if this might ever happen).
  • If anything needs to be added to a namelist description, please submit another PR.

There are many changes described in PR text, but I have confirmed this branch reproduces itself.

@JessicaMeixner-NOAA JessicaMeixner-NOAA merged commit 3952826 into NOAA-EMC:develop Jan 16, 2024
20 checks passed
@mickaelaccensi
Copy link
Collaborator Author

@mickaelaccensi I'm approving this PR, however, it would be great if you could follow up on two things still:

* Confirm the two spots in w3src4md will not ever be divide by zero (a follow-up PR for work around is sufficient if this might ever happen).

It should not happen but I'll confirm that with Fabrice

* If anything needs to be added to a namelist description, please submit another PR.

I'm working on it, I just need some additional information from Fabrice, then I'll make a PR

There are many changes described in PR text, but I have confirmed this branch reproduces itself.

@JessicaMeixner-NOAA
Copy link
Collaborator

Thanks @mickaelaccensi !

@mickaelaccensi
Copy link
Collaborator Author

I confirm that division by CG=0 should never happen

@JessicaMeixner-NOAA
Copy link
Collaborator

I confirm that division by CG=0 should never happen

Thank you!!!

miguelsolanocordoba added a commit to wavespotter/WW3 that referenced this pull request Apr 19, 2024
* Bugfix - initialised VD and VS to zero in w3srcemd. (NOAA-EMC#1037)

* More efficient test for binary files in matrix.comp (NOAA-EMC#1035)

* Tidy up of pre-processor directives and unused variables in w3srcemd.F90 (NOAA-EMC#1010)

* Correct typo in w3srcemd.F90 pre-processor directive. (NOAA-EMC#1039)

* minor bugfix for matrix grepping on keywords (NOAA-EMC#1049)

* Stop masking group 1 output where icec > icen (NOAA-EMC#1019)

* Doxygen documentation added, 8th subset.(NOAA-EMC#1046)

* NC4 ,F90 ,XX0 switches removed from ww3_tp2.19 regtest (NOAA-EMC#1054)

* CI:  Fix for Intel scripts. GNU scripts updated. (NOAA-EMC#1064)

* correct the computation of QP parameter, add QKK output parameter, change UST scale factor (NOAA-EMC#1050)

* correct issue with ww3_multi when requesting restart2 and using nml file instead of inp file (NOAA-EMC#1070)

* correct calendar for track netcdf output (NOAA-EMC#1079)

* Fix missing mod_def.ww3 file in multigrid regression tests for track output (NOAA-EMC#1091)

* STAB3: fix cmake build for ST4 or ST3 (NOAA-EMC#1086)

* new feature to output out_grd.ww3, out_pnt.ww3 and mod_def.ww3 both in binary and ascii format using switch ASCII. (NOAA-EMC#1089)

* Update local unit number arrays (NDS, MDS) to be same size of array defined in w3odatmd (size=15). Also, defined unit numbers for NDS(14) and NDS(15). (NOAA-EMC#1098)

* Removed code referencing PHIOC in output section for PHICE in ww3_ounf (NOAA-EMC#1093)

* implementation of the GQM (Gaussian Quadrature Method) to replace the DIA in NL1 or NL2. (NOAA-EMC#1083)

* update logic to ensure you are not accessing uninitialized dates (NOAA-EMC#1114)

* Initialised S and D arrays in W3SDB1 before potential early return if zero energy. (NOAA-EMC#1115)

* ww3_ounp.F90:  x/y units attribute corrected from 'm' to 'km' (NOAA-EMC#1088)

* Bugfix: Assign unit numbers to ASCII gridded/point output in multi-grid mode. (NOAA-EMC#1118)

* correct bugs to run correctly GQM implementation (NOAA-EMC#1127)

* Adding documentation to w3iopo() in preparation for code for NOAA-EMC#682. (NOAA-EMC#1131)

* NCEP regtest module updates: uses spack-stack/1.5.0, includes scotch/7.0.4 (NOAA-EMC#1137)

* Minor update to ncep regtests (NOAA-EMC#1138)

* Updated intel workflow to install oneapi compilers from new location. (NOAA-EMC#1157)

* Add unit test for points I/O code. (NOAA-EMC#1158)

* Update Intel CI (relocate /usr/local; ensure intel-oneapi-mpi; use ubuntu-latest) (NOAA-EMC#1161)

* remove lookup table for ST4 to speed up computation and clean up the ST4 code (NOAA-EMC#1124)

Co-authored-by: Fabrice Ardhuin <[email protected]>

* initialize USSP_WN for mod_def (NOAA-EMC#1165)

* Introduce IC4M8 and IC4M9 to WW3 (NOAA-EMC#1176)

* clean up and add ST4 variables (NOAA-EMC#1181)

* w3fld1md.F90: fix divide by zero in CRIT2 parameter (NOAA-EMC#1184)

* ww3_prnc.F90: fix out-of-scope grid index write statement (NOAA-EMC#1185)

* Bugfix: address potential divide-by-zero in APPENDTAIL (NOAA-EMC#1188)

Co-authored-by: Denise Worthen <[email protected]>

* Provide initial drying of cells with depth < ZLIM for SMC grid. (NOAA-EMC#1192)

* Output OMP threading info to screen when running ww3_shel/ww3_multi compiled with the OMPG switch. Also fixes truncation of build.log when running run_cmake_build. (NOAA-EMC#1191)

* Added screen output showing number of threads when OMP enabled.

* update build to get more info in logs (NOAA-EMC#46)

---------

Co-authored-by: Jessica Meixner <[email protected]>

* update run_cmake_test to catch build errors and exit (NOAA-EMC#1194)

* fix merge conflicts

* Fix gustiness bug, as suggst by Pieter

* Change USTARsigma to WAM implementation

---------

Co-authored-by: Chris Bunney <[email protected]>
Co-authored-by: Mickael Accensi <[email protected]>
Co-authored-by: Benoit Pouliot <[email protected]>
Co-authored-by: Matthew Masarik <[email protected]>
Co-authored-by: Ghazal-Mohammadpour <[email protected]>
Co-authored-by: Jessica Meixner <[email protected]>
Co-authored-by: Biao Zhao <[email protected]>
Co-authored-by: Edward Hartnett <[email protected]>
Co-authored-by: Alex Richert <[email protected]>
Co-authored-by: Fabrice Ardhuin <[email protected]>
Co-authored-by: W. Erick Rogers <[email protected]>
Co-authored-by: Denise Worthen <[email protected]>
Co-authored-by: Camille Teicheira <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[W3SRC4] WHITECAP not initialised before use
3 participants