windfarm_opt=1 issue with MPI #1865
Comments
MarcImberger
added a commit
to MarcImberger/WRF
that referenced
this issue
Dec 20, 2023
TYPE: bug fix
KEYWORDS: wind, fitch, standing thrust coefficient, broadcast, mpi
SOURCE: Marc Imberger, Technical University of Denmark, Denmark
DESCRIPTION OF CHANGES:
Problem:
Curently, the standing thrust coefficient above cut-out wind speed ('stc2') is not
broadcasted from the master node in the fitch wind farm parameterization scheme.
This has the consequence that any wind turbine that
(1) exceeds the cut-out wind speed at hub-height and that (2) is not located in a WRF
tile associated with the MPI master process will have a non-deterministic standing thrust
coefficient. Since stc2 is also not initialized within the code, this value could
be in principle anything (I guess this also depends on how the compiler deals with
unitinialized arrays). This affects mostly 'thrcof' which
modifies the horizontal velocity tendencies and can thus yield to
unphysical feedback to the rest of the model or even model crash (CFL violations).
Solution:
Adding stc2 to the list of variables to broadcast (see l. 516 in WRF/phys/module_wind_fitch.F)
using CALL wrf_dm_bcast_real(stc2,nt) addresses this issue.
ISSUE: For use when this PR closes an issue.
This PR fixes issue wrf-model#1865
LIST OF MODIFIED FILES:
- WRF/phys/module_wind_fitch.F
TESTS CONDUCTED:
- Flawed broadcasting could in principle be demonstrated by printing the values of 'stc2' by each MPI process.
- Issue has been found by running a simulation with windfarm_opt=1 for a short episode where wind speeds at hub-height
exceed cut-out wind speed and where the turbine is not located in the tile assocated with the master process.
In my case, model crash due unreasonably high velocity tendencies (CFL violation) has been observed.
- In less extreme cases (e.g. when stc2 contains 0 as described in issue wrf-model#1865), 'thrcof' is set to 0 causing
no turbine feedback to the velocity tendencies (while typical standing thrust coefficients are larger 0)
RELEASE NOTE:
Add missing wrf_dm_bcast_real for stc2 in fitch parameterization scheme affecting standing
thrust coefficients for wind speeds at turbine hub-heights above cut-out.
|
Hi sael9740, Nice detective work! I have stumbled across this effect as well (in my case, stc2 was not set to 0 but some other random value causing my model to crash under stormy conditions. I could imagine that if it's set to 0 or no is maybe related to the compiler/compiler flags). I made a PR request about this. Marc |
weiwangncar
pushed a commit
that referenced
this issue
Dec 21, 2023
TYPE: bug fix
KEYWORDS: wind, fitch, standing thrust coefficient, broadcast, mpi
SOURCE: Marc Imberger, Technical University of Denmark, Denmark
DESCRIPTION OF CHANGES:
Problem:
Curently, the standing thrust coefficient above cut-out wind speed ('stc2') is not broadcasted from the master node in the fitch wind farm parameterization scheme. This has the consequence that any wind turbine that (1) exceeds the cut-out wind speed at hub-height and that (2) is not located in a WRF
tile associated with the MPI master process will have a non-deterministic standing thrust coefficient. Since stc2 is also not initialized within the code, this value could be in principle anything (I guess this also depends on how the compiler deals with uninitialialized arrays). This affects mostly 'thrcof' which
modifies the horizontal velocity tendencies and can thus yield to unphysical feedback to the rest of the model or even model crash (CFL violations).
Solution:
Adding stc2 to the list of variables to broadcast (see l. 516 in WRF/phys/module_wind_fitch.F)
using CALL wrf_dm_bcast_real(stc2,nt) addresses this issue.
ISSUE: For use when this PR closes an issue.
This PR fixes issue #1865
LIST OF MODIFIED FILES:
- WRF/phys/module_wind_fitch.F
TESTS CONDUCTED:
- Flawed broadcasting could in principle be demonstrated by printing the values of 'stc2' by each MPI process.
- Issue has been found by running a simulation with windfarm_opt=1 for a short episode where wind speeds at hub-height exceed cut-out wind speed and where the turbine is not located in the tile assocated with the master process. In my case, model crash due unreasonably high velocity tendencies (CFL violation) has been observed.
- In less extreme cases (e.g. when stc2 contains 0 as described in issue #1865), 'thrcof' is set to 0 causing no turbine feedback to the velocity tendencies (while typical standing thrust coefficients are larger 0)
- The code passed regression tests.
RELEASE NOTE: Add missing wrf_dm_bcast_real for stc2 in fitch parameterization scheme affecting standing thrust coefficients for wind speeds at turbine hub-heights above cut-out.
vlakshmanan-scala
pushed a commit
to scala-computing/WRF
that referenced
this issue
Apr 4, 2024
TYPE: bug fix
KEYWORDS: wind, fitch, standing thrust coefficient, broadcast, mpi
SOURCE: Marc Imberger, Technical University of Denmark, Denmark
DESCRIPTION OF CHANGES:
Problem:
Curently, the standing thrust coefficient above cut-out wind speed ('stc2') is not broadcasted from the master node in the fitch wind farm parameterization scheme. This has the consequence that any wind turbine that (1) exceeds the cut-out wind speed at hub-height and that (2) is not located in a WRF
tile associated with the MPI master process will have a non-deterministic standing thrust coefficient. Since stc2 is also not initialized within the code, this value could be in principle anything (I guess this also depends on how the compiler deals with uninitialialized arrays). This affects mostly 'thrcof' which
modifies the horizontal velocity tendencies and can thus yield to unphysical feedback to the rest of the model or even model crash (CFL violations).
Solution:
Adding stc2 to the list of variables to broadcast (see l. 516 in WRF/phys/module_wind_fitch.F)
using CALL wrf_dm_bcast_real(stc2,nt) addresses this issue.
ISSUE: For use when this PR closes an issue.
This PR fixes issue wrf-model#1865
LIST OF MODIFIED FILES:
- WRF/phys/module_wind_fitch.F
TESTS CONDUCTED:
- Flawed broadcasting could in principle be demonstrated by printing the values of 'stc2' by each MPI process.
- Issue has been found by running a simulation with windfarm_opt=1 for a short episode where wind speeds at hub-height exceed cut-out wind speed and where the turbine is not located in the tile assocated with the master process. In my case, model crash due unreasonably high velocity tendencies (CFL violation) has been observed.
- In less extreme cases (e.g. when stc2 contains 0 as described in issue wrf-model#1865), 'thrcof' is set to 0 causing no turbine feedback to the velocity tendencies (while typical standing thrust coefficients are larger 0)
- The code passed regression tests.
RELEASE NOTE: Add missing wrf_dm_bcast_real for stc2 in fitch parameterization scheme affecting standing thrust coefficients for wind speeds at turbine hub-heights above cut-out.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
While working with the wind farm code phys/module_wind_fitch.F I noticed that the variable
stc2variable is initialized for rank 0 but not broadcast to the other ranks:WRF/phys/module_wind_fitch.F
Lines 506 to 529 in 21c7214
I confirmed that
stc2was not initialized correctly for the other ranks (in my case they had all zero values in this array).Tracking this variable I found that the only place it is used is here in the
dragcofroutine:WRF/phys/module_wind_fitch.F
Lines 307 to 308 in 21c7214
Therefore
tkecofwould end up being 0 for all ranks except for rank 0 if the condition on line 307 was met. This seems like an issue that could have easily been overlooked but I'm sure that the developer intended to broadcaststc2after it was initialized by rank 0.The text was updated successfully, but these errors were encountered: