Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Regression test fv3_ccpp_rrfs_v1beta_debug crashes - commented out in rt*.conf #174

Closed
climbfuji opened this issue Jul 22, 2020 · 1 comment
Closed

Regression test fv3_ccpp_rrfs_v1beta_debug crashes - commented out in rt*.conf #174

climbfuji opened this issue Jul 22, 2020 · 1 comment
Labels
bug Something isn't working

Comments

@climbfuji
Copy link
Collaborator

Description

Regression test fv3_ccpp_rrfs_v1beta_debug was added in PR #173. This test runs on cheyenne.intel, but crashes on hera.intel, hera.gnu and cheyenne.gnu. The error message is:

115: mp_thompson_post_run: ttendlim applied 5/2048 times at timestep 1
136: forrtl: error (182): floating invalid - possible uninitialized real/complex variable.
136: Image              PC                Routine            Line        Source
136: fv3.exe            00000000080B9B7F  Unknown               Unknown  Unknown
136: libpthread-2.17.s  00002B012962D5F0  Unknown               Unknown  Unknown
136: fv3.exe            00000000042C1AF4  noahmp_glacier_ro        1375  module_sf_noahmp_glacier.f90
136: fv3.exe            00000000042BFE16  noahmp_glacier_ro        1127  module_sf_noahmp_glacier.f90
136: fv3.exe            00000000042B57D8  noahmp_glacier_ro         565  module_sf_noahmp_glacier.f90
136: fv3.exe            00000000042B39B7  noahmp_glacier_ro         297  module_sf_noahmp_glacier.f90
136: fv3.exe            0000000003B80B43  noahmpdrv_mp_noah         664  sfc_noahmp_drv.f
136: fv3.exe            00000000030ADB0F  ccpp_fv3_rrfs_v1b         990  ccpp_FV3_RRFS_v1beta_physics_cap.F90
136: fv3.exe            0000000002D936CF  ccpp_static_api_m         287  ccpp_static_api.F90
136: fv3.exe            0000000002D9A4F5  ccpp_driver_mp_cc         152  CCPP_driver.F90
136: libiomp5.so        00002B0129C27A43  __kmp_invoke_micr     Unknown  Unknown

This could be simply a numerical instability, or an uninitialized variable.

To Reproduce:

Uncomment the regression test fv3_ccpp_rrfs_v1beta_debug in rt.conf or rt_gnu.conf and run it.
@climbfuji climbfuji added the bug Something isn't working label Jul 22, 2020
@climbfuji climbfuji mentioned this issue Jul 22, 2020
Merged
DusanJovic-NOAA pushed a commit that referenced this issue Jul 23, 2020
@climbfuji
Prepare for SRW App release (update develop from NCAR dtc/develop 202…
c6b3566 
…0/07/21) (#173)

- new regression test configurations for `fv3_ccpp_rrfs_v1beta` and `fv3_ccpp_rrfs_v1beta_debug` (required for SRW App release)
- added `fv3_ccpp_rrfs_v1beta` test to various `rt*.conf` files; note: not adding `fv3_ccpp_rrfs_v1beta_debug` because this test crashes on hera with both Intel and GNU (it runs on Cheyenne with Intel but not with GNU) - see issue #174
- moved the flake regression test in `rt_gnu.conf` from the DEBUG to the PROD section (this was a mistake not caught in the previous PR when flake was merged)
- remove all `tests/Compile*.log` log files
@climbfuji
Copy link
Collaborator Author

This was fixed in a previous commit thanks to @panll's investigation.

epic-cicd-jenkins pushed a commit that referenced this issue Apr 17, 2023
Merge pull request #174 from NOAA-EMC/update_EMC_post_hash
ef31814 
Update EMC_post hash to top of DTC_post branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@climbfuji