Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix: inconsistent arrays sizes for NDS/MDS. #1098

Merged

Conversation

ukmo-ccbunney
Copy link
Collaborator

Pull Request Summary

Fixes inconsistent array size bug in w3initmd.

Description

Fixes bug where new NDS array in W3ODAT has been expanded to size 15 to accommodate new ASCII output unit numbers (see #1089) , but some local arrays in other parts of the code are still of size 13.

Also, unit numbers for NDS(14) and NDS(15) were not explicitly defined (as far as I can tell).

Issue(s) addressed

Fixes: #1097

Commit Message

Update local unit number arrays (NDS, MDS) to be same size of array defined in w3odatmd (size=15). Also, defined unit numbers for NDS(14) and NDS(15).

Check list

Testing

  • How were these changes tested? TBC
  • Are the changes covered by regression tests? (If not, why? Do new tests need to be added?)
  • Have the matrix regression tests been run (if yes, please note HPC and compiler)?
  • Please indicate the expected changes in the regression test output, (Note the list of known non-identical tests.)
  • Please provide the summary output of matrix.comp (matrix.Diff.txt, matrixCompFull.txt and matrixCompSummary.txt):

W3ODATMD (15). Also, defined unit numbers for NDS(14) and NDS(15)
@MatthewMasarik-NOAA
Copy link
Collaborator

Thank you, Chris! I'll put in the regtests.

Copy link
Collaborator

@mickaelaccensi mickaelaccensi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for this PR Chris

@ukmo-ccbunney
Copy link
Collaborator Author

thanks for this PR Chris

No worries - sorry for missing on the review!

@MatthewMasarik-NOAA
Copy link
Collaborator

The tests and comp came back. The results are close to a clean report, but looks like there is something small awry, maybe a transposed index, with respect to ww3_tp2.6.

matrix.comp summary

**********************************************************************          
********************* non-identical cases ****************************          
**********************************************************************          
mww3_test_03/./work_PR3_UNO_MPI_e                     (1 files differ)          
mww3_test_03/./work_PR2_UQ_MPI_e                     (1 files differ)           
mww3_test_03/./work_PR2_UNO_MPI_d2                     (8 files differ)         
mww3_test_03/./work_PR1_MPI_d2                     (13 files differ)            
mww3_test_03/./work_PR3_UNO_MPI_d2_c                     (15 files differ)      
mww3_test_03/./work_PR3_UQ_MPI_d2_c                     (15 files differ)       
mww3_test_03/./work_PR3_UNO_MPI_d2                     (14 files differ)        
mww3_test_03/./work_PR2_UQ_MPI_d2                     (16 files differ)         
mww3_test_03/./work_PR3_UNO_MPI_e_c                     (1 files differ)        
mww3_test_03/./work_PR3_UQ_MPI_d2                     (16 files differ)         
mww3_test_09/./work_MPI_ASCII                     (0 files differ)              
ww3_tp2.10/./work_MPI_OMPH                     (5 files differ)                 
ww3_tp2.16/./work_MPI_OMPH                     (4 files differ)                 
ww3_tp2.6/./work_ST4_ASCII                     (2 files differ)                 
ww3_ufs1.3/./work_a                     (3 files differ)                        
                                                                                
**********************************************************************          
************************ identical cases *****************************          
**********************************************************************

The two files that differ are out_grd.ww3.txt and out_pnt.ww3.txt. Looking at the line counts, it looks like the two output's got switched largely (also, with a difference of 200 lines -- 5647/5847, 344227/344427).

[hfe04]$ wc -l ww3-???/regtests/ww3_tp2.6/work*ASCII/out_???.*.txt
    5647 ww3-dev/regtests/ww3_tp2.6/work_ST4_ASCII/out_grd.ww3.txt
  344427 ww3-dev/regtests/ww3_tp2.6/work_ST4_ASCII/out_pnt.ww3.txt
  344227 ww3-fix/regtests/ww3_tp2.6/work_ST4_ASCII/out_grd.ww3.txt
    5847 ww3-fix/regtests/ww3_tp2.6/work_ST4_ASCII/out_pnt.ww3.txt
  700148 total

@MatthewMasarik-NOAA
Copy link
Collaborator

Note, the binaries (out_grd.ww3, out_pnt.ww3) are identical.

@ukmo-ccbunney
Copy link
Collaborator Author

The tests and comp came back. The results are close to a clean report, but looks like there is something small awry, maybe a transposed index, with respect to ww3_tp2.6.

matrix.comp summary

**********************************************************************          
********************* non-identical cases ****************************          
**********************************************************************          
mww3_test_03/./work_PR3_UNO_MPI_e                     (1 files differ)          
mww3_test_03/./work_PR2_UQ_MPI_e                     (1 files differ)           
mww3_test_03/./work_PR2_UNO_MPI_d2                     (8 files differ)         
mww3_test_03/./work_PR1_MPI_d2                     (13 files differ)            
mww3_test_03/./work_PR3_UNO_MPI_d2_c                     (15 files differ)      
mww3_test_03/./work_PR3_UQ_MPI_d2_c                     (15 files differ)       
mww3_test_03/./work_PR3_UNO_MPI_d2                     (14 files differ)        
mww3_test_03/./work_PR2_UQ_MPI_d2                     (16 files differ)         
mww3_test_03/./work_PR3_UNO_MPI_e_c                     (1 files differ)        
mww3_test_03/./work_PR3_UQ_MPI_d2                     (16 files differ)         
mww3_test_09/./work_MPI_ASCII                     (0 files differ)              
ww3_tp2.10/./work_MPI_OMPH                     (5 files differ)                 
ww3_tp2.16/./work_MPI_OMPH                     (4 files differ)                 
ww3_tp2.6/./work_ST4_ASCII                     (2 files differ)                 
ww3_ufs1.3/./work_a                     (3 files differ)                        
                                                                                
**********************************************************************          
************************ identical cases *****************************          
**********************************************************************

The two files that differ are out_grd.ww3.txt and out_pnt.ww3.txt. Looking at the line counts, it looks like the two output's got switched largely (also, with a difference of 200 lines -- 5647/5847, 344227/344427).

[hfe04]$ wc -l ww3-???/regtests/ww3_tp2.6/work*ASCII/out_???.*.txt
    5647 ww3-dev/regtests/ww3_tp2.6/work_ST4_ASCII/out_grd.ww3.txt
  344427 ww3-dev/regtests/ww3_tp2.6/work_ST4_ASCII/out_pnt.ww3.txt
  344227 ww3-fix/regtests/ww3_tp2.6/work_ST4_ASCII/out_grd.ww3.txt
    5847 ww3-fix/regtests/ww3_tp2.6/work_ST4_ASCII/out_pnt.ww3.txt
  700148 total

Hi @MatthewMasarik-NOAA

Yes - I am seeing those differences too. I couldn't understand it at first, but I believe that this is correct behaviour and the output in develop is incorrect.

The output for out_pnt.ww3.txt in develop contains lines like this:

TIME, FLOGRD:    20100801          10 F T F F T F F F T F F T F F T F F F T F F T F F T F F F T F F T
 MAPSTA:           1           1           1           1           1           1           1          
 HS:   0.00000000       0.00000000       0.00000000       0.00000000       0.00000000       0.00000000
 WLM:   0.00000000       0.00000000       0.00000000       0.00000000       0.00000000       0.0000000
 T02:  0.711681604      0.711681604      0.711681604      0.711681604      0.711681604      0.71168160
 T0M1:  0.711681604      0.711681604      0.711681604      0.711681604      0.711681604      0.7116816

which is actually the ASCII output from W3IOGO, not W3IOPO.

I think this is happening in develop because the unit numbers for NDS(14) and NDS(15) are not defined and are probably (by chance) have the same value resulting in out_pnt.ww3.txt and out_grd.ww3.txt being opened with the same unit number.

Looking at the new code in W3IOPOMD that writes out the ASCII fields, this branch is writing out the correct output.

Can @mickaelaccensi confirm?

@MatthewMasarik-NOAA
Copy link
Collaborator

Hi @ukmo-ccbunney

Ah, that's very interesting. Then we should see those differences drop if we ran your fix against itself. I'll get that going now. Aside from also getting back confirmation from Mickael, does that sound like a good path?

@ukmo-ccbunney
Copy link
Collaborator Author

ukmo-ccbunney commented Oct 13, 2023

Hi @ukmo-ccbunney

Ah, that's very interesting. Then we should see those differences drop if we ran your fix against itself. I'll get that going now.

That's my theory at least! :)

Aside from also getting back confirmation from Mickael, does that sound like a good path?

Sounds sensible!

@MatthewMasarik-NOAA
Copy link
Collaborator

Awesome, and thanks for your very quick detective work! :)

@mickaelaccensi
Copy link
Collaborator

good catch guys and sorry to have introduced a bug with the ascii feature !

@MatthewMasarik-NOAA
Copy link
Collaborator

The matrix.comp results look mostly good. With the exception of hitting a few (0 files differ), though they seem to be an issue encountered before, where links of anl.grbtxt from input_* to work_* get broken in one of the runs. This will be good to merge, though I'm going to try to see if I can figure out how to fix the issue this morning briefly. Otherwise, I'll make an issue to return to it, so we can keep the queue moving.

matrixCompSummary.txt
matrixCompFull.txt
matrixDiff.txt

**********************************************************************
********************* non-identical cases ****************************
**********************************************************************
mww3_test_02/./work_PR1_c                     (0 files differ)
mww3_test_02/./work_PR2_UNO_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_a                     (0 files differ)
mww3_test_02/./work_PR1_MPI_d                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_b_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_c_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_c                     (0 files differ)
mww3_test_02/./work_PR2_UNO_MPI_d                     (0 files differ)
mww3_test_02/./work_PR2_UNO_d                     (0 files differ)
mww3_test_02/./work_PR2_UQ_MPI_a                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_a_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_d_c                     (0 files differ)
mww3_test_02/./work_PR1_d                     (0 files differ)
mww3_test_02/./work_PR2_UQ_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_b                     (0 files differ)
mww3_test_02/./work_PR3_UQ_a                     (0 files differ)
mww3_test_02/./work_PR1_a                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_d                     (0 files differ)
mww3_test_02/./work_PR3_UNO_a_c                     (0 files differ)
mww3_test_02/./work_PR2_UQ_MPI_b                     (0 files differ)
mww3_test_02/./work_PR3_UNO_d                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_c_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_a_c                     (0 files differ)
mww3_test_02/./work_PR2_UQ_a                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_b                     (0 files differ)
mww3_test_02/./work_PR3_UNO_b_c                     (0 files differ)
mww3_test_02/./work_PR1_b                     (0 files differ)
mww3_test_02/./work_PR2_UNO_b                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_c_c                     (0 files differ)
mww3_test_02/./work_PR2_UQ_d                     (0 files differ)
mww3_test_02/./work_PR3_UQ_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_a                     (0 files differ)
mww3_test_02/./work_PR1_MPI_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_b                     (0 files differ)
mww3_test_02/./work_PR1_MPI_b                     (0 files differ)
mww3_test_02/./work_PR3_UQ_a_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_b_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_d_c                     (0 files differ)
mww3_test_02/./work_PR1_MPI_a                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_d_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_d                     (0 files differ)
mww3_test_02/./work_PR3_UQ_d                     (0 files differ)
mww3_test_02/./work_PR3_UNO_d_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_c_c                     (0 files differ)
mww3_test_02/./work_PR2_UNO_MPI_b                     (0 files differ)
mww3_test_02/./work_PR3_UNO_MPI_c                     (0 files differ)
mww3_test_02/./work_PR2_UQ_b                     (0 files differ)
mww3_test_02/./work_PR3_UNO_c                     (0 files differ)
mww3_test_02/./work_PR3_UNO_a                     (0 files differ)
mww3_test_02/./work_PR2_UQ_MPI_d                     (0 files differ)
mww3_test_02/./work_PR2_UQ_MPI_c                     (0 files differ)
mww3_test_02/./work_PR2_UNO_a                     (0 files differ)
mww3_test_02/./work_PR2_UNO_MPI_c                     (0 files differ)
mww3_test_02/./work_PR2_UNO_MPI_a                     (0 files differ)
mww3_test_02/./work_PR3_UQ_b_c                     (0 files differ)
mww3_test_02/./work_PR3_UQ_MPI_b                     (0 files differ)
mww3_test_03/./work_PR3_UQ_MPI_e_c                     (1 files differ)
mww3_test_03/./work_PR3_UNO_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UQ_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UNO_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UNO_MPI_d2                     (16 files differ)
mww3_test_03/./work_PR1_MPI_d2                     (10 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2_c                     (13 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2_c                     (15 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2                     (9 files differ)
mww3_test_03/./work_PR2_UQ_MPI_d2                     (13 files differ)
mww3_test_03/./work_PR3_UNO_MPI_e_c                     (1 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2                     (15 files differ)
mww3_test_09/./work_MPI_ASCII                     (0 files differ)
ww3_ta1/./work_UPD0F_O                     (0 files differ)
ww3_ta1/./work_UPD2_U_cap                     (0 files differ)
ww3_ta1/./work_UPD3_U                     (0 files differ)
ww3_ta1/./work_UPD5_U_cap                     (0 files differ)
ww3_ta1/./work_UPD6_O                     (0 files differ)
ww3_ta1/./work_UPD2_U                     (0 files differ)
ww3_ta1/./work_UPD5_O                     (0 files differ)
ww3_ta1/./work_UPD5_U                     (0 files differ)
ww3_ta1/./work_UPD3_U_cap                     (0 files differ)
ww3_ta1/./work_UPD3_O                     (0 files differ)
ww3_ta1/./work_UPD6_U                     (0 files differ)
ww3_ta1/./work_UPD2_O                     (0 files differ)
ww3_ta1/./work_UPD6_U_cap                     (0 files differ)
ww3_tp2.10/./work_MPI_OMPH                     (7 files differ)
ww3_tp2.16/./work_MPI_OMPH                     (4 files differ)
ww3_tp2.6/./work_ST4_ASCII                     (0 files differ)
ww3_ufs1.3/./work_a                     (3 files differ)
**********************************************************************
************************ identical cases *****************************
**********************************************************************

Copy link
Collaborator

@MatthewMasarik-NOAA MatthewMasarik-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code Review

Pass

Testing

Pass

matrix.comp output

**********************************************************************
********************* non-identical cases ****************************
**********************************************************************
mww3_test_03/./work_PR1_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UQ_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UNO_MPI_e                     (1 files differ)
mww3_test_03/./work_PR2_UNO_MPI_d2                     (17 files differ)
mww3_test_03/./work_PR1_MPI_d2                     (12 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2_c                     (18 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2_c                     (16 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2                     (12 files differ)
mww3_test_03/./work_PR2_UQ_MPI_d2                     (16 files differ)
mww3_test_03/./work_PR3_UQ_MPI_e                     (1 files differ)
mww3_test_03/./work_PR3_UNO_MPI_e_c                     (1 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2                     (16 files differ)
mww3_test_09/./work_MPI_ASCII                     (0 files differ)
ww3_tp2.10/./work_MPI_OMPH                     (7 files differ)
ww3_tp2.16/./work_MPI_OMPH                     (4 files differ)
ww3_tp2.6/./work_ST4_ASCII                     (0 files differ)
ww3_ufs1.3/./work_a                     (3 files differ)
**********************************************************************
************************ identical cases *****************************
**********************************************************************

matrixCompSummary.txt
matrixCompFull.txt
matrixDiff.txt

Here we can see the expected non-b4b, and additionally:

  • There are (0 files differ) for the added ASCII tests,
  • Potentially one or two extra (1 files differ) for mww3_test_03 tests.

I discussed these extra non-b4b's with @JessicaMeixner-NOAA and we assessed:

  • The ASCII (0 files differ) are not a concern and is likely an easy matrix* fix.
  • The mww3_test_03 tests do seem to fluctuate on the number of non-b4b tests. Here the differing file is a restart (restart001.low0). This could potentially be an initialization issue, regardless it's likely not due to this PR. So to move forward I will make an issue noting it so we can merge this PR, and be aware to keep an eye on the mww3_test_03 tests.

@MatthewMasarik-NOAA MatthewMasarik-NOAA merged commit d148d09 into NOAA-EMC:develop Oct 16, 2023
20 checks passed
@MatthewMasarik-NOAA
Copy link
Collaborator

Thanks @ukmo-ccbunney for the real-time bug checking!

@ukmo-ccbunney ukmo-ccbunney deleted the bugfix/NDS_bounds_issue1097 branch October 17, 2023 08:18
miguelsolanocordoba added a commit to wavespotter/WW3 that referenced this pull request Apr 19, 2024
* Bugfix - initialised VD and VS to zero in w3srcemd. (NOAA-EMC#1037)

* More efficient test for binary files in matrix.comp (NOAA-EMC#1035)

* Tidy up of pre-processor directives and unused variables in w3srcemd.F90 (NOAA-EMC#1010)

* Correct typo in w3srcemd.F90 pre-processor directive. (NOAA-EMC#1039)

* minor bugfix for matrix grepping on keywords (NOAA-EMC#1049)

* Stop masking group 1 output where icec > icen (NOAA-EMC#1019)

* Doxygen documentation added, 8th subset.(NOAA-EMC#1046)

* NC4 ,F90 ,XX0 switches removed from ww3_tp2.19 regtest (NOAA-EMC#1054)

* CI:  Fix for Intel scripts. GNU scripts updated. (NOAA-EMC#1064)

* correct the computation of QP parameter, add QKK output parameter, change UST scale factor (NOAA-EMC#1050)

* correct issue with ww3_multi when requesting restart2 and using nml file instead of inp file (NOAA-EMC#1070)

* correct calendar for track netcdf output (NOAA-EMC#1079)

* Fix missing mod_def.ww3 file in multigrid regression tests for track output (NOAA-EMC#1091)

* STAB3: fix cmake build for ST4 or ST3 (NOAA-EMC#1086)

* new feature to output out_grd.ww3, out_pnt.ww3 and mod_def.ww3 both in binary and ascii format using switch ASCII. (NOAA-EMC#1089)

* Update local unit number arrays (NDS, MDS) to be same size of array defined in w3odatmd (size=15). Also, defined unit numbers for NDS(14) and NDS(15). (NOAA-EMC#1098)

* Removed code referencing PHIOC in output section for PHICE in ww3_ounf (NOAA-EMC#1093)

* implementation of the GQM (Gaussian Quadrature Method) to replace the DIA in NL1 or NL2. (NOAA-EMC#1083)

* update logic to ensure you are not accessing uninitialized dates (NOAA-EMC#1114)

* Initialised S and D arrays in W3SDB1 before potential early return if zero energy. (NOAA-EMC#1115)

* ww3_ounp.F90:  x/y units attribute corrected from 'm' to 'km' (NOAA-EMC#1088)

* Bugfix: Assign unit numbers to ASCII gridded/point output in multi-grid mode. (NOAA-EMC#1118)

* correct bugs to run correctly GQM implementation (NOAA-EMC#1127)

* Adding documentation to w3iopo() in preparation for code for NOAA-EMC#682. (NOAA-EMC#1131)

* NCEP regtest module updates: uses spack-stack/1.5.0, includes scotch/7.0.4 (NOAA-EMC#1137)

* Minor update to ncep regtests (NOAA-EMC#1138)

* Updated intel workflow to install oneapi compilers from new location. (NOAA-EMC#1157)

* Add unit test for points I/O code. (NOAA-EMC#1158)

* Update Intel CI (relocate /usr/local; ensure intel-oneapi-mpi; use ubuntu-latest) (NOAA-EMC#1161)

* remove lookup table for ST4 to speed up computation and clean up the ST4 code (NOAA-EMC#1124)

Co-authored-by: Fabrice Ardhuin <[email protected]>

* initialize USSP_WN for mod_def (NOAA-EMC#1165)

* Introduce IC4M8 and IC4M9 to WW3 (NOAA-EMC#1176)

* clean up and add ST4 variables (NOAA-EMC#1181)

* w3fld1md.F90: fix divide by zero in CRIT2 parameter (NOAA-EMC#1184)

* ww3_prnc.F90: fix out-of-scope grid index write statement (NOAA-EMC#1185)

* Bugfix: address potential divide-by-zero in APPENDTAIL (NOAA-EMC#1188)

Co-authored-by: Denise Worthen <[email protected]>

* Provide initial drying of cells with depth < ZLIM for SMC grid. (NOAA-EMC#1192)

* Output OMP threading info to screen when running ww3_shel/ww3_multi compiled with the OMPG switch. Also fixes truncation of build.log when running run_cmake_build. (NOAA-EMC#1191)

* Added screen output showing number of threads when OMP enabled.

* update build to get more info in logs (NOAA-EMC#46)

---------

Co-authored-by: Jessica Meixner <[email protected]>

* update run_cmake_test to catch build errors and exit (NOAA-EMC#1194)

* fix merge conflicts

* Fix gustiness bug, as suggst by Pieter

* Change USTARsigma to WAM implementation

---------

Co-authored-by: Chris Bunney <[email protected]>
Co-authored-by: Mickael Accensi <[email protected]>
Co-authored-by: Benoit Pouliot <[email protected]>
Co-authored-by: Matthew Masarik <[email protected]>
Co-authored-by: Ghazal-Mohammadpour <[email protected]>
Co-authored-by: Jessica Meixner <[email protected]>
Co-authored-by: Biao Zhao <[email protected]>
Co-authored-by: Edward Hartnett <[email protected]>
Co-authored-by: Alex Richert <[email protected]>
Co-authored-by: Fabrice Ardhuin <[email protected]>
Co-authored-by: W. Erick Rogers <[email protected]>
Co-authored-by: Denise Worthen <[email protected]>
Co-authored-by: Camille Teicheira <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

NDS: Array size mismatch?
3 participants