Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Appendtail (called by w3fld1 and w3fld2) can get NaNs from variable "avg" #1159

Closed
DeniseWorthen opened this issue Jan 2, 2024 · 5 comments · Fixed by ufs-community/ufs-weather-model#2085
Labels
bug Something isn't working

Comments

@DeniseWorthen
Copy link
Contributor

DeniseWorthen commented Jan 2, 2024

Describe the bug

The variable AVG in SR appendtail can be zero and results in NaNs being returned to w3fld2 in array SPC2. This is triggered when implementing wave-ice coupling (using CICE) in UWM and CESM. When a NaN is returned, fld2 will fail-safe with the code

    IF (.not.((CD .LT. 0.01).AND.(CD .GT. 0.0005)).or. .not.(NO_ERR)) THEN
      !Fail safe to bulk

In debug mode, the model will fail but typically at a point longer than the UWM debug RTs, which are 3 hours (ie, the model fails after 8 hours).

This may or may not be caught by WW3 standalone RTs. @MatthewMasarik-NOAA What is the longest debug RT test that utilizes FLD1 or FLD2?

To Reproduce

Set up a case w/ wave-ice coupling in UWM and run the debug case> 8hours. The model will fail w/ float invalid. In non-debug, the model does not fail but produces the log message

Calculated Wind/Cd: ',UREF,CD,UST

where both CD and UST are NaN.

Expected behavior

Division using avg should not occur when avg==0.0.

@NickSzapiro-NOAA

@DeniseWorthen DeniseWorthen added the bug Something isn't working label Jan 2, 2024
@MatthewMasarik-NOAA
Copy link
Collaborator

@DeniseWorthen for standalone WW3 we don't currently have any RT's that run in debug mode.

I'll check to see how long regtests with FLD1 or FLD2 run for.

As an aside, testing PDLIB code recently I'm also hitting a floating point (divide by zero) error when running in debug, occurring in w3fld1md.F90. This was just found so I'm actively looking into it.

@DeniseWorthen
Copy link
Contributor Author

DeniseWorthen commented Jan 2, 2024

@MatthewMasarik-NOAA Ok, just FYI with wave-ice coupling the NaN has been found for both structured and unstructured cases.

And no standalone RTs are done in debug? That seems very odd.

In any case, the code in appendtail should IMHO not allow a division by AVG if AVG can ever be zero.

@MatthewMasarik-NOAA
Copy link
Collaborator

@MatthewMasarik-NOAA Ok, just FYI with wave-ice coupling the NaN has been found for both structured and unstructured cases.

Thanks for identifying this @DeniseWorthen. It's helpful to know it's in both structured/unstructured with wave-ice coupling.

And no standalone RTs are done in debug? That seems very odd.

Correct, right now no regtests are done in debug. The topic of testing is currently coming up with some other WW3 work. I've made a note to bring up testing in debug mode.

In any case, the code in appendtail should IMHO not allow a division by AVG if AVG can ever be zero.

Agreed.

@junwang-noaa
Copy link
Contributor

@MatthewMasarik-NOAA May I ask what the timeline is to get the fix committed in ww3? Thanks

@JessicaMeixner-NOAA
Copy link
Collaborator

@junwang-noaa according to the commit queue it will be 1/24: https://github.com/ufs-community/ufs-weather-model/wiki/Commit-Queue although it sounds like it might get moved up to this week potentially.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
4 participants