Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge 4.2.2 #50

Merged
merged 38 commits into from
Jan 17, 2021
Merged

Merge 4.2.2 #50

merged 38 commits into from
Jan 17, 2021

Conversation

letmaik
Copy link

@letmaik letmaik commented Jan 16, 2021

No CMake or CI changes needed.

jamiebresch and others added 30 commits September 16, 2020 15:36
…mat=2 (wrf-model#1287)

TYPE: bug fix

KEYWORDS: WRFDA, alpha_hydrometeors, ep_format=2

SOURCE: Jamie Bresch (NCAR)

DESCRIPTION OF CHANGES:
Problem:
Only rootproc reads in the data, but the broadcasting from rootproc was missing for qice, qsnow and qgraup.
Non-root processors will have qrain values for qice, qsnow and qgraup.

Solution:
add call wrf_dm_bcast_real(temp3d, ijk)

LIST OF MODIFIED FILES:
M var/da/da_setup_structures/da_setup_flow_predictors_ep_format2.inc

TESTS CONDUCTED:

RELEASE NOTE: Big fix for WRFDA incorrect ensemble perturbation values for qice, qsnow and qgraup when alpha_hydrometeors=true and ep_format=2.
TYPE: bug fix

KEYWORDS: WRFDA, use_obs_errfac, max_ext_its > 1

SOURCE: Jamie Bresch (NCAR)

DESCRIPTION OF CHANGES:
Problem:
CWB reported seeing obs errors keep decreasing with each outerloop.

Solution:
Add a check for outerloop number, so that when use_obs_errfac=true and errfac.dat exists in the working directory, observation error factors are applied only at the first outerloop.

LIST OF MODIFIED FILES:
M var/da/da_minimisation/da_get_innov_vector.inc

TESTS CONDUCTED:

RELEASE NOTE: Bug fix for WRFDA's incorrectly applying errfac.dat at each outerloop when use_obs_errfac=true and max_ext_its > 1.
TYPE: bug fix

KEYWORDS: chem, drydep, wesely

SOURCE: Lukas Pilz (Heidelberg University)

DESCRIPTION OF CHANGES:
Problem:
A namelist option incompatibility between chem_opt (16) and gas_drydep_opt (1) leads to trying to access a variable
that is not available in memory. This leads to unpredictable behaviors. When choosing a tracer-only chem_opt, numgas 
is 0. If gas_drydep_opt is 1 (the default), in the Wesely scheme initialization (dep_init), the field `dvj` is initialized with 
size numgas (here 0). Accessing the variable `dvj` is an error.

Solution:
A fatal error was added for when the Wesely scheme initialization is called with numgas = 0.

This fix is in `chem/chemics_init.F` and not `share/module_check_a_mundo.F` because checking the namelist options 
would require some tricky-to-maintain hardcoding.

LIST OF MODIFIED FILES: 
chem/chemics_init.F

TESTS CONDUCTED:
1. Run with chem_opt = 16 and gas_drydep_opt = 1 fails appropriately
2. Run with chem_opt = 16 and gas_drydep_opt = 0 runs as it should
3. The jenkins testing is OK.

RELEASE NOTE: For WRF-Chem, a namelist option incompatibility between chem_opt (16) and gas_drydep_opt (1) leads to trying to access a variable that is not available in memory. This leads to unpredictable behaviors. A fatal error was added for when the Wesely scheme initialization is called with numgas = 0.
TYPE: bug fix

KEYWORDS: Deng shcu, resolved hydrometeors, non-active updrafts, precipitation units, variable initialization, limit NT

SOURCE: Pedro A. Jimenez and Tim Juliano (NCAR/RAL), Eric Grimit and Mattew Wiley (Vaisala), and Xin-Zhong Liang
(University of Maryland)

DESCRIPTION OF CHANGES:
Problem:

The resolved hydrometeors are added to the unresolved hydrometeors inside Deng's parameterization. The radiation
driver adds the resolved hydrometeors to the unresolved ones causing to double counting the effects of the resolved
hydrometeors.
The unresolved clouds are not properly removed when the updraft is not active
Incorrect precipitation units
Incorrect initialization of variable radiusc. The variable is initialized inside an #if construct for the 1D model
If dt < 18 s NT is larger than 100 causing the code to look for values outside of array bounds for ainckfsa variable
Solution:

Do not add the resolved hydrometeors to the unresolved ones inside Deng's parameterization
Set cloud fraction and the mixing ratio of the unresolved hydrometeors to 0 if the updraft is not active.
Remove the conversion factor that was leading to the incorrect units.
Initialize radiusc outside of the #if construct so radiusc is always initialized.
Limit the maximum value of DT to 100
The following figure illustrates the impact of corrections 1 and 2. The time series of the mixing ratio now properly goes
to 0 if the updraft is not active.
image

LIST OF MODIFIED FILES:
M phys/module_shcu_deng.F

TESTS CONDUCTED:

We have run simulations with and without the fixes. The figure above illustrates that the clouds are now adequately
removed if there are not active updrafts.
Jenkins tests are all passing
RELEASE NOTE: Several fixes introduced to the Deng's shallow cumulus parameterization: 1) Do not add the resolved hydrometeors to the unresolved ones to avoid double counting, 2) remove clouds if the updraft is not active, 3) correct conversion factor for precipitation, 4) correct initialization of radiusc variable, and 5) limit variable NT to avoid array out of bounds.
…odel#1280)

TYPE: bug fix

KEYWORDS: ideal case, initialization

SOURCE: Tim Raupach (UNSW CCRC)

DESCRIPTION OF CHANGES:
Problem:
In ideal case initialization routines, incorrect indices were used in the calculation of ph_1 after a perturbation bubble was added.

Impact:
The calculation of perturbation geopotential ph_1 (perturbation geopotential) and Z_BASE (idealized base state height) 
were affected.
1. If hybrid coordinate option was selected, (hybrid_opt = 2), the entire ph_1 initialization is incorrect due to the offset
in the indexing of c1h and c2h (half levels vs full levels).
2. If hybrid_opt = 0, since the 1d variables are initialized kms:kme, there is no impact.

Solution:
The vertical indices were corrected in the calculations of ph_1 in the idealized initialization routine.

ISSUE:

LIST OF MODIFIED FILES:
M       dyn_em/module_initialize_ideal.F

TESTS CONDUCTED: 
1. The jenkins testing is all pass.

RELEASE NOTE: In the ideal case initialization routines, incorrect indices were used in the calculation of ph_1 after a perturbation bubble was added. For the idealized cases that do not use the hybrid vertical coordinate by default, there is no impact at all. The ideal cases with a hill had the correct hybrid formulation.
FARMS correctly detects unresolved clouds

TYPE: bug fix

KEYWORDS: FARMS, effective radius of unresolved hydrometeors, cloud fraction

SOURCE: Ju-Hye Kim and Pedro A. Jimenez (NCAR/RAL), and Jimy Dudhia (NCAR/MMM)

DESCRIPTION OF CHANGES:
Problem:
The previous version of FARMS did not correctly account for the radiative effects of unresolved clouds, because the FARMS calculates the surface irradiances using both cloud water contents and cloud effective radii. In previous version, cloud water contents came from both resolved and unresolved clouds, on the other hand, effective radii came only from resolved clouds.

Solution:
The climatological effective radii are assigned to unresolved clouds of liquid, ice and snow species, and are specified as the effective radii of clouds in microphysics schemes (e.g. mp=2) that do not provide any effective radii. The fix is also valid for Thompson MPs, WSMMPs, and WDMMPs by adjusting the background effective radii of these schemes. In addition to this, we improved calculations of surface irradiance by adding cloud fraction to the FARMS scheme.

ISSUE: N/A

LIST OF MODIFIED FILES:
M phys/module_mp_thompson.F
M phys/module_mp_wdm5.F
M phys/module_mp_wdm6.F
M phys/module_mp_wdm7.F
M phys/module_mp_wsm3.F
M phys/module_mp_wsm5.F
M phys/module_mp_wsm6.F
M phys/module_mp_wsm7.F
M phys/module_physics_init.F
M phys/module_ra_farms.F
M phys/module_radiation_driver.F

TESTS CONDUCTED:

Do mods fix problem? How can that be demonstrated, and was that test conducted?
Yes. The downward short wave flux (GHI) will increase in the cloud region with unresolved clouds by reduced scatterings due to increased cloud size. This was tested in the WRF-Solar model.

Are the Jenkins tests all passing?
We did't do the Jenkins tests.

RELEASE NOTE: Bug fix to account for the effective radius of the unresolved hydrometeors by FARMS.
TYPE: bug fix

KEYWORDS: mpi, real, bug

SOURCE: Marc Honnorat (EXWEXs)

DESCRIPTION OF CHANGES:
Problem:
When running real.exe on multiple processes with MPI, one or more process occasionally crashes in setup_physics_suite 
(in share/module_check_a_mundo.F). This has been linked to wrf_dm_initialize non-blocking MPI. 

The real.exe occasionally crashes in setup_physics_suite (in share/module_check_a_mundo.F#L2640) because the latter 
uses model_config_rec % physics_suite, which on some machines is not initialized. The behavior is as if the broadcast of 
model_config_rec performed just before in main/real_em.F#L124 had not been received by all processes.

I had never seen this bug before, it has only happened on one machine (an Intel-based cluster using Intel-MPI and ifort).

The current fix makes sure that all processes are well synced before proceeding with setup_physics_suite. It solves the 
issue on my machine. Since this is immediately after reading in the namelist, no performance issues are expected as this 
read and broadcast of the namelist occurs only once per real / WRF / ndown run.

Solution:
An MPI barrier is added at the end of wrf_dm_initialize to force all of the processes to be synchronized before checking 
the namelist consistency.

This is a simplification of PR wrf-model#1268, which had extra white space.

ISSUE:
Fixes wrf-model#1267

LIST OF MODIFIED FILES:
M external/RSL_LITE/module_dm.F

TESTS CONDUCTED:
1. On the only machine were I have seen the bug occur, this change fixes the problem. No other test was conducted since I couldn't reproduce the bug on another setup.
2. Jenkins testing is all PASS.

RELEASE NOTES: When running real.exe on multiple processes with MPI, one or more processes occasionally crash in setup_physics_suite (in share/module_check_a_mundo.F). This has been traced to the fact that wrf_dm_initialize is non-blocking from an MPI point of view.  The problem is intermittent and has only happened on one machine (an Intel-based cluster using Intel-MPI and ifort). An MPI barrier has been added at the end of wrf_dm_initialize to force all processes to be synchronized before checking namelist consistency.
TYPE: bug fix

KEYWORDS: HALO, h_mom_adv_order

SOURCE: Ted Mansell (NOAA/NSSL)

DESCRIPTION OF CHANGES: 
Problem:
The halo exchange (used during distributed memory communication) had inconsistent use of `h_mom_adv_order`. There 
are locations when the namelist option to check should have `h_sca_adv_order`, and even a couple cases where the
code should have tested both. Consequently, the MPI results did not match for different number of patches or processor 
counts if `h_mom_adv_order=3` and `h_sca_adv_order=5`. This problem only occurs when the advection order differs 
between the momentum and scalar variables.

Solution:
These changes (using the correct variable to test, or using both `h_mom_adv_order` and `h_sca_adv_order`) fix the 
problem of non-reproducibility on different core counts (where non-reproducibility means "wrong answers"). Also, a few of 
the halo exchanges in solve_em need to check both the momentum and scalar flags, since both types are communicated 
in the subroutine call.

LIST OF MODIFIED FILES:
dyn_em/module_first_rk_step_part1.F
dyn_em/module_first_rk_step_part2.F
dyn_em/solve_em.F

TESTS CONDUCTED:
1. Jenkins testing is all PASS

Tested ideal case (em_quarter_ss) on 4 vs 1 processes, with the following advection orders:
   - h_mom_adv_order=3
   - h_sca_adv_order=5
2. Without changes, diffwrf found differences. 
3. After changes the wrfout files match exactly. (Also tested for h_mom_adv_order=5, h_sca_adv_order=3)

RELEASE NOTES: The halo exchange (used during distributed memory communication) had inconsistent use of h_mom_adv_order. There are locations in the ARW dynamics solver when the namelist option to check should have been h_sca_adv_order, and even a couple cases where the IF tests should have tested both options (both h_mom_adv_order and h_sca_adv_order). Consequently, the model results did not match bit-for-bit for differing numbers processors (when the advection order differed between the momentum and scalar variables). These IF-test changes (using the correct namelist option to compare, or possibly using both the h_mom_adv_order and h_sca_adv_order options) fix the problem of non-reproducibility due to the advection order. This problem only occurs when the advection order differs between the momentum and scalar variables.
…#1251)

TYPE: bug fix

KEYWORDS: gfortran/10

SOURCE: Srikanth Yalavarthi (Marvell Semiconductor)

DESCRIPTION OF CHANGES: 
Problem:
Due to argument inconsistencies, the WRF code will not compile with gfortran/10. 

Here are some examples of incorrect Fortran argument usages:

```
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (rank-1 and scalar)
wrf_io.f:7605:71:

 7605 |                                                    ,i1,i2,j1,j2,k1,k2 )
      |                                                                       1
```
```
Error: Type mismatch in argument ‘field’ at (1); passed INTEGER(4) to REAL(8)
wrf_io.f:7744:49:

 2538 |     stat = NF_GET_ATT_INT (DH%NCID,NF_GLOBAL,Element,Buffer)
      |                                                     2
......
 7744 |     stat = NF_GET_ATT_INT(NCID,VarID,'FieldType',FType)
      |                                                 1
```
```
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (scalar and rank-3)
field_routines.f:158:52:

  110 |     stat = NF_GET_VARA_INT(NCID,VarID,VStart,VCount,Data)
      |                                                    2
......
  158 |     stat = NF_GET_VARA_INT(NCID,VarID,VStart,VCount,Buffer)
      |                                                    1
```
```
Error: Rank mismatch in argument ‘fileindex’ at (1) (scalar and rank-1)
io_grib1.f90:685:27:

  685 |   CALL gr1_fill_eta_levels(fileinfo(DataHandle)%fileindex(:), FileFd(DataHandle), &
      |                           1
```
```
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(4)/REAL(4)).
module_internal_header_util.f:1592:39:

 1592 |                            DataHandle, Data, Count, code )
      |                                       1
......
 1654 |                            DataHandle, Data, Count, code )
      |                                       2
```
```
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (rank-1 and scalar)
module_io_int_read.f:129:39:

  129 |     call mpi_file_read_at(ifd, offset, tmp, 1, &
      |                                       1
......
  573 |     call mpi_file_read_at(ifd, offset, tmp, num, &
      |                                       2
```
```
Error: Type mismatch between actual argument at (1) and actual argument at (2) (REAL(8)/COMPLEX(8)).
zmfm1b.f90:74:38:

   72 |       call zmf3kb ( lot, ido, l1, na, c, jump, inc, ch, 1, lot, wa(iw) )
      |                                      2
   73 |     else if ( nbr == 4 ) then
   74 |       call zmf3kb ( lot, ido, l1, na, ch, 1, lot, c, jump, inc, wa(iw) )
      |                                      1
```
The errors in the WRF infrastructure code can be eventually updated. However, the code from the FFTPACK library 
will be left AS-IS. We are going to initially address all of the problems in a temporary fashion.

Temporary solution:
A compiler flag was added for the gfortran compiler to allow mismatched arguments, which enables the WRF system
to build. The new flag is only for versions of gfortran 10 and later. This change was introduced in the configure script,
which modifies all gfortran stanzas.

Permanent fix:
A complete fix requires changes in the Fortran sources for BOZ declarations and to fix mismatches in subroutine 
arguments.

ISSUE: 
Fixes wrf-model#1250

LIST OF MODIFIED FILES: 
M   Makefile
M   arch/configure.defaults
M   configure

TESTS CONDUCTED: 
1. Successfully built the master branch with GCC-10 on RPi4 Ubuntu-20.04 machine
2. Logic tested, where only GNU 10 and above are impacted.
3. Other compilers are not affected.
4. Jenkins testing is OK.
5. The DA variants have been tested also:  I could build WRF / WRFPLUS / 3DVAR / 4DVAR
```
[00:01] [syalavarthi@sparrow WRF]$./configure wrfda && ./compile all_wrfvar && ls var/build/*.exe
var/build/da_advance_time.exe         var/build/da_tune_obs_hollingsworth1.exe  var/build/gen_be_addmean.exe                var/build/gen_be_diags.exe       var/build/gen_be_hist.exe          var/build/gen_be_stage2_1dvar.exe     var/build/gen_mbe_stage2.exe
var/build/da_bias_airmass.exe         var/build/da_tune_obs_hollingsworth2.exe  var/build/gen_be_cov2d.exe                  var/build/gen_be_diags_read.exe  var/build/gen_be_stage0_gsi.exe    var/build/gen_be_stage2_gsi.exe
var/build/da_bias_scan.exe            var/build/da_update_bc.exe                var/build/gen_be_cov2d3d_contrib.exe        var/build/gen_be_ensmean.exe     var/build/gen_be_stage0_wrf.exe    var/build/gen_be_stage2a.exe
var/build/da_bias_sele.exe            var/build/da_update_bc_ad.exe             var/build/gen_be_cov3d.exe                  var/build/gen_be_ensrf.exe       var/build/gen_be_stage1.exe        var/build/gen_be_stage3.exe
var/build/da_bias_verif.exe           var/build/da_verif_grid.exe               var/build/gen_be_cov3d2d_contrib.exe        var/build/gen_be_ep1.exe         var/build/gen_be_stage1_1dvar.exe  var/build/gen_be_stage4_global.exe
var/build/da_rad_diags.exe            var/build/da_verif_obs.exe                var/build/gen_be_cov3d3d_bin3d_contrib.exe  var/build/gen_be_ep2.exe         var/build/gen_be_stage1_gsi.exe    var/build/gen_be_stage4_regional.exe
var/build/da_tune_obs_desroziers.exe  var/build/da_wrfvar.exe                   var/build/gen_be_cov3d3d_contrib.exe        var/build/gen_be_etkf.exe        var/build/gen_be_stage2.exe        var/build/gen_be_vertloc.exe
[00:10] [syalavarthi@sparrow WRF]$

[06:40] [syalavarthi@sparrow WRFPLUS]$./configure wrfplus && ./compile wrfplus && ls main/*.exe
main/wrfplus.exe

[07:16] [syalavarthi@sparrow WRF]$./configure 4dvar && ./compile all_wrfvar && ls var/build/*.exe
var/build/da_advance_time.exe         var/build/da_tune_obs_hollingsworth1.exe  var/build/gen_be_addmean.exe                var/build/gen_be_diags.exe       var/build/gen_be_hist.exe          var/build/gen_be_stage2_1dvar.exe     var/build/gen_mbe_stage2.exe
var/build/da_bias_airmass.exe         var/build/da_tune_obs_hollingsworth2.exe  var/build/gen_be_cov2d.exe                  var/build/gen_be_diags_read.exe  var/build/gen_be_stage0_gsi.exe    var/build/gen_be_stage2_gsi.exe
var/build/da_bias_scan.exe            var/build/da_update_bc.exe                var/build/gen_be_cov2d3d_contrib.exe        var/build/gen_be_ensmean.exe     var/build/gen_be_stage0_wrf.exe    var/build/gen_be_stage2a.exe
var/build/da_bias_sele.exe            var/build/da_update_bc_ad.exe             var/build/gen_be_cov3d.exe                  var/build/gen_be_ensrf.exe       var/build/gen_be_stage1.exe        var/build/gen_be_stage3.exe
var/build/da_bias_verif.exe           var/build/da_verif_grid.exe               var/build/gen_be_cov3d2d_contrib.exe        var/build/gen_be_ep1.exe         var/build/gen_be_stage1_1dvar.exe  var/build/gen_be_stage4_global.exe
var/build/da_rad_diags.exe            var/build/da_verif_obs.exe                var/build/gen_be_cov3d3d_bin3d_contrib.exe  var/build/gen_be_ep2.exe         var/build/gen_be_stage1_gsi.exe    var/build/gen_be_stage4_regional.exe
var/build/da_tune_obs_desroziers.exe  var/build/da_wrfvar.exe                   var/build/gen_be_cov3d3d_contrib.exe        var/build/gen_be_etkf.exe        var/build/gen_be_stage2.exe        var/build/gen_be_vertloc.exe
```

RELEASE NOTE: Due to subroutine and function argument inconsistencies, the WRF code will not compile with gfortran/10.  The fix introduces a new compiler flag for all gfortran stanzas (`-fallow-argument-mismatch -fallow-invalid-boz`).
…1286)

TYPE: bug fix

KEYWORDS: occasional instabilities, sfclayrev, look up tables

SOURCE: Pedro A. Jimenez (NCAR/RAL)

DESCRIPTION OF CHANGES:
Problem:
For some occasions the model turns unstable, and this was found to be originating due to the surface layer option 1, with
the upper boundary values of the look up tables.

Solution:
Avoiding using the look up tables of the integrated similarity functions for the last tabulated value.

LIST OF MODIFIED FILES:
M phys/module_sf_sfclayrev.F

TESTS CONDUCTED:
After introducing the fix the model no longer turns unstable in the specific instances noted.
The jenkins tests are all PASS.

RELEASE NOTE: A minor fix (excluding use of the upper bounding value of the look-up table) was introduced to avoid occasional instabilities in the surface layer option 1.
)

TYPE: Enhancement

KEYWORDS: Noah-MP-Crop, specific-crop, parameters, photosynthesis, stomata

SOURCE: Zhe Zhang (University of Saskatchewan)

DESCRIPTION OF CHANGES: The current release WRF-Crop option 1, (Liu Xing's model), uses generic-crop parameters in stomata subroutine, to calculate photosynthesis and stomata resistance, regardless of crop type. Also some parameters in the crop parameters in MPTABLE.TBL, the first and second columns for corn and soybean are not consistent with the values used in Xing's paper. This pull request fixes the parameter values for these two columns in the crop parameter section in MPTABLE.TBL, as well as adds specific-crop photosynthesis-stomata parameters in this section.

LIST OF MODIFIED FILES:
phys/module_sf_noahmplsm.F
phys/module_sf_noahmpdrv.F
run/MPTABLE.TBL

TESTS CONDUCTED:

Coupled WRF-Crop simulation is conducted on Cheyenne. The results seem reasonable in terms of crop LAI and biomass.
Presented these results with the Noah-MP LSM group meeting with Dr. Fei Chen.
Jenkins testing is all PASS.

RELEASE NOTE: Incorporate specific-crop parameters for photosynthesis and stomata subroutine in Noah-MP-Crop. The current release WRF-Crop option 1, (Liu Xing's model), uses generic-crop parameters in stomata subroutine, to calculate photosynthesis and stomata resistance, regardless of crop type. Also some parameters in the crop parameters in MPTABLE.TBL, the first and second columns for corn and soybean are not consistent with the values used in Xing's paper. This update fixes the parameter values for these two columns in the crop parameter section in MPTABLE.TBL, as well as adds specific-crop photosynthesis-stomata parameters in this section.
Zhang, Z., Barlage, M., Chen, F., Li, Y., Helgason, W., Xu, X., et al. (2020). Joint modeling of crop and irrigation in the Central United States using the Noah‐MP land surface model. Journal of Advances in Modeling Earth Systems. https://doi.org/10.1029/2020MS002159
…el#1300)

TYPE: bug fix

KEYWORDS: integer kind

SOURCE: internal

DESCRIPTION OF CHANGES:
Problem:
The nine calls to io_int_loc had inconsistent kinds of integer for the actual and dummy arguments. On every machine and
compiler previously, the integer kinds for MPI_offset and c_int64_t were identical. On the raspberry pi (using the 
default apt-get openmpi and gnu fortran), these are not identical.

Solution:
As the integers are being shared through a C interface, that integer kind was selected.

LIST OF MODIFIED FILES:
modified: external/io_int/module_io_int_read.F90

TESTS CONDUCTED: 
1. Without the mods, the WRF model did not build on a raspberry pi. With the mods, the code builds and runs. Here testing
on a raspberry pi includes the distributed memory, so the MPI_offset variable would be in use.

```
pi@raspberrypi:/media/pi/gill/WRF/test/em_quarter_ss $ mpirun -np 1 ./wrf.exe ; tail rsl.out.0000
 starting wrf task            0  of            1
Timing for main: time 0001-01-01_00:00:24 on domain   1:    0.79946 elapsed seconds
Timing for main: time 0001-01-01_00:00:36 on domain   1:    0.79981 elapsed seconds
Timing for main: time 0001-01-01_00:00:48 on domain   1:    0.80066 elapsed seconds
Timing for main: time 0001-01-01_00:01:00 on domain   1:    0.79946 elapsed seconds
Timing for main: time 0001-01-01_00:01:12 on domain   1:    0.80392 elapsed seconds
Timing for main: time 0001-01-01_00:01:24 on domain   1:    0.80236 elapsed seconds
Timing for main: time 0001-01-01_00:01:36 on domain   1:    0.80006 elapsed seconds
Timing for main: time 0001-01-01_00:01:48 on domain   1:    0.79996 elapsed seconds
Timing for main: time 0001-01-01_00:02:00 on domain   1:    0.80133 elapsed seconds
d01 0001-01-01_00:02:00 wrf: SUCCESS COMPLETE WRF
```
```
pi@raspberrypi:/media/pi/gill/WRF/test/em_quarter_ss $ mpirun -np 2 ./wrf.exe ; tail rsl.out.0000
 starting wrf task            1  of            2
 starting wrf task            0  of            2
Timing for main: time 0001-01-01_00:00:24 on domain   1:    0.53475 elapsed seconds
Timing for main: time 0001-01-01_00:00:36 on domain   1:    0.53516 elapsed seconds
Timing for main: time 0001-01-01_00:00:48 on domain   1:    0.53412 elapsed seconds
Timing for main: time 0001-01-01_00:01:00 on domain   1:    0.53324 elapsed seconds
Timing for main: time 0001-01-01_00:01:12 on domain   1:    0.53357 elapsed seconds
Timing for main: time 0001-01-01_00:01:24 on domain   1:    0.53547 elapsed seconds
Timing for main: time 0001-01-01_00:01:36 on domain   1:    0.53507 elapsed seconds
Timing for main: time 0001-01-01_00:01:48 on domain   1:    0.53406 elapsed seconds
Timing for main: time 0001-01-01_00:02:00 on domain   1:    0.53477 elapsed seconds
d01 0001-01-01_00:02:00 wrf: SUCCESS COMPLETE WRF
```
```
pi@raspberrypi:/media/pi/gill/WRF/test/em_quarter_ss $ mpirun -np 3 ./wrf.exe ; tail rsl.out.0000
 starting wrf task            0  of            3
 starting wrf task            1  of            3
 starting wrf task            2  of            3
Timing for main: time 0001-01-01_00:00:24 on domain   1:    0.48709 elapsed seconds
Timing for main: time 0001-01-01_00:00:36 on domain   1:    0.48533 elapsed seconds
Timing for main: time 0001-01-01_00:00:48 on domain   1:    0.48523 elapsed seconds
Timing for main: time 0001-01-01_00:01:00 on domain   1:    0.48553 elapsed seconds
Timing for main: time 0001-01-01_00:01:12 on domain   1:    0.48532 elapsed seconds
Timing for main: time 0001-01-01_00:01:24 on domain   1:    0.48635 elapsed seconds
Timing for main: time 0001-01-01_00:01:36 on domain   1:    0.48445 elapsed seconds
Timing for main: time 0001-01-01_00:01:48 on domain   1:    0.48409 elapsed seconds
Timing for main: time 0001-01-01_00:02:00 on domain   1:    0.48494 elapsed seconds
d01 0001-01-01_00:02:00 wrf: SUCCESS COMPLETE WRF

```
```
pi@raspberrypi:/media/pi/gill/WRF/test/em_quarter_ss $ mpirun -np 4 ./wrf.exe ; tail rsl.out.0000
 starting wrf task            3  of            4
 starting wrf task            0  of            4
 starting wrf task            1  of            4
 starting wrf task            2  of            4
Timing for main: time 0001-01-01_00:00:24 on domain   1:    0.65011 elapsed seconds
Timing for main: time 0001-01-01_00:00:36 on domain   1:    0.64889 elapsed seconds
Timing for main: time 0001-01-01_00:00:48 on domain   1:    0.64934 elapsed seconds
Timing for main: time 0001-01-01_00:01:00 on domain   1:    0.64886 elapsed seconds
Timing for main: time 0001-01-01_00:01:12 on domain   1:    0.65009 elapsed seconds
Timing for main: time 0001-01-01_00:01:24 on domain   1:    0.64658 elapsed seconds
Timing for main: time 0001-01-01_00:01:36 on domain   1:    0.64700 elapsed seconds
Timing for main: time 0001-01-01_00:01:48 on domain   1:    0.64807 elapsed seconds
Timing for main: time 0001-01-01_00:02:00 on domain   1:    0.64591 elapsed seconds
d01 0001-01-01_00:02:00 wrf: SUCCESS COMPLETE WRF
```
2. Jenkins testing is all PASS.

RELEASE NOTES: An inconsistency in the declaration of integer kinds was discovered in the seldom-used routine that processes binary formatted data. In one routine the values were declared as an MPI type (and integer kind mpi_offset_type). That argument was passed to a routine that declared the integers as c_int64_t (which is part of the standard technique now to be used when working with C). These two integer kinds must have been the same 8-byte size for all of the x86_64 Linux machines that we have used. These integer kinds are however different for at least some gfortran / openmpi combinations on ARM processors (specifically found on a raspberry pi). There is no impact to existing users. If the code previously worked (built), this changes nothing.
Fix for high sulfate concentrations with  biomass_burn_opt = 2

TYPE: bug fix

KEYWORDS: chem,biomassburn,plumerise,sulfate

SOURCE: Neeldip Barman (IIT Guwahati)

DESCRIPTION OF CHANGES:
Problem:
Namelist option biomass_burn_opt = 2 when used with chem_opt=202 produces very high so4 concentration and so4 plumes reaching near the model top. This further leads to very large optical depths.

Namelist option biomass_burn_opt = 2 does not take sulfate emissions as inputs in registry, but sulfate emission inputs have been defined in module_mosaic_addemiss.F

Solution:
Setting IF ( p_ebu_sulf .gt. 1) aem_so4 = bburn_mosaic_f(n)*ebu(i,k,j,p_ebu_sulf) inplace of aem_so4 = bburn_mosaic_f(n)*ebu(i,k,j,p_ebu_sulf) 

LIST OF MODIFIED FILES: 
module_mosaic_addemiss.F

TESTS CONDUCTED:
 1. chem_opt = 202 with the code modification 

RELEASE NOTE: sulf emissions are not defined for biomass_burn_opt = 2 (MOZART option) in the registry, but sulf emission inputs have been defined in module_mosaic_addemiss.F, which is used for any MOZART option coupled to MOSAIC
TYPE: bug fix

KEYWORDS: FARMS, Microphysics, module_check_a_mundo.F

SOURCE: Ju Hye Kim and Pedro Jimenez (NCAR/RAL)

DESCRIPTION OF CHANGES:

Problem:
FARMS can work correctly with microphysics schemes that do not provide effective radii of clouds. But, current WRF model displays an ERROR message and stops when the 'swint_opt is 2' and 'use_mp_re is not 1'.

Solution:
We remove this constraint from ‘module_check_a_mundo.F’.

LIST OF MODIFIED FILES:
M share/module_check_a_mundo.F

TESTS CONDUCTED:
1. When FARMS and a microphysics scheme that do not provide effective radii of clouds (ex: mp=2) are used together, the model does not stop anymore.
2. All jenkins tests are passing.

RELEASE NOTE: Fixed an incorrect stop in the model when FARMS radiation is used with microphysics that do not provide effective cloud radii.
TYPE: bug fix

KEYWORDS: saturation vapor pressure, Morrison double-moment scheme

SOURCE: Hugh Morrison (NCAR)

DESCRIPTION OF CHANGES:
Problem:
Current code gave unrealistically high saturation vapor pressures at T < -80 C.

Solution:
"function polysvp" in module_mp_morr_two_moment_aero.F and module_mp_morr_two_moment.F are modified to correct the problem.

LIST OF MODIFIED FILES:
M phys/module_mp_morr_two_moment_aero.F
M phys/module_mp_morr_two_moment.F

TESTS CONDUCTED:
Idealized tests conduced by Morrison, the mods fix the problem.
Jenkins tests have passed.

RELEASE NOTE: Fix a problem with function polyqvp which gave unrealistically high saturation vapor pressures at T < -80 C.
TYPE: bug fix

KEYWORDS: irrigation, time selection, day selection, bug fix

SOURCE: Arianna Valmassoi (Uni-Bonn)

DESCRIPTION OF CHANGES:
Problem:
The irrigation schemes weren't working when the date selection was across two different years: i.e. irr_end_julianday < irr_start_julianday. Same problem was found for active hours across the local 00 time: i.e. irr_start_hour+irr_num_hours>23

Solution:
Fixed by adding/modifying the if statements

LIST OF MODIFIED FILES:
M phys/module_irrigation.F

TESTS CONDUCTED:
Do mods fix problem? yes
Are the Jenkins tests all passing? Passed.

RELEASE NOTE: Bug fixes for temporal selection that spans across different years and different local days.
…rf-model#1313)

TYPE: bug fix

KEYWORDS: sea-ice fraction option, QSFC calculation, MYJSFC, QNSESFC

SOURCE: Xin-Zhong Liang

DESCRIPTION OF CHANGES:
Problem: Incorrect parentheses in weighted average

Solution:
Fixed parentheses
Effect: Test by Kevin Manning indicates small effect.

ISSUE: For use when this PR closes an issue.
Fixes wrf-model#1297 (part)

LIST OF MODIFIED FILES:
M phys/module_surface_driver.F

TESTS CONDUCTED:
Test by Kevin Manning shows small impact.
Jenkins test (TBD)

RELEASE NOTE:
Fix incorrect parentheses in weighted average for QSFC when sea-ice fraction is used with MYJSFC and QNSESFC surface layer physics.
wrf-model#1259)

TYPE: bug fix

KEYWORDS: surface heat flux, use_theta_m, moist theta, non-local flux, 3D TKE

SOURCE: Matthias Göbel (University of Innsbruck)

DESCRIPTION OF CHANGES: The conversion of surface heat fluxes from dry to moist theta in WRF is inconsistent. This leads to an unclosed dry theta budget at the lowest grid point. The surface sensible heat flux in the routines `vertical_diffusion_2` and `vertical_diffusion_implicit` should be multiplied with <img src="https://latex.codecogs.com/svg.latex?1+\frac{R_{v}}{R_{d}}q"  /> as is derived below.
Using Reynold's decomposition and averaging we can show for the perturbation moist theta:
<img src="https://latex.codecogs.com/svg.latex?\theta_{m}'=(\theta(1+q\alpha))'=\theta(1+q\alpha)-\overline{\theta(1+q\alpha)}=\theta-\overline{\theta}+\alpha(q\theta-\overline{q\theta})=\\=\theta'+\alpha\left[(\overline{\theta}+\theta')(\overline{q}+q')-\overline{\theta}\overline{q}-\overline{\theta'q'}\right]=\theta'(1+\alpha\overline{q})+\alpha\left[\overline{\theta}q'+\theta'q'-\overline{\theta'q'}\right]"  /> 
with water vapor mixing ratio q and
 <img src="https://latex.codecogs.com/svg.latex?\alpha=\frac{R_{v}}{R_{d}}\approx1.61"  />

This yields for the converted surface heat flux:
<img src="https://latex.codecogs.com/svg.latex?\overline{w'\theta_{m}'}_0=(1+\alpha\overline{q}_0)\overline{w'\theta'}_0+\alpha\overline{\theta}_0\,\overline{w'q'}_0+\alpha\overline{w'\theta'q'}_0\approx(1+\alpha\overline{q}_0)\overline{w'\theta'}_0+\alpha\overline{\theta}_0\,\overline{w'q'}_0"  />
The triple correlation in the equation above cannot be easily estimated in WRF, but judging from my tests it probably has a negligible impact.
Instead of the given equation, WRF uses:
<img src="https://latex.codecogs.com/svg.latex?\overline{w'\theta_{m}'}_0=\overline{w'\theta'}_0+\alpha\overline{\theta}_0\overline{w'q'}_0 "  />
We therefore multiplied the surface heat flux with <img src="https://latex.codecogs.com/svg.latex?1+\alpha\overline{q}_0" />, when `use_theta_m=1`.
For `km_opt=5`, also the contribution from the non-local heat flux is multiplied with a correction factor.

LIST OF MODIFIED FILES: dyn_em/module_diffusion_em.F 

TESTS CONDUCTED: checked the dry theta budget of a simulation with use_theta_m=1: simple idealized LES once with model-computed surface heat flux and once with prescribed heat fluxes, RRTMG radiation, no microphysics, zero eddy diffusivities. Changed the code to output advection, SGS diffusion and radiation tendencies for dry theta (regardless of use_theta_m and without changing the tendencies used by the model). These forcing tendencies are averaged online over 30 minutes. The sum of the forcing terms is plotted against the actual tendency of dry theta  (change of instantaneous dry theta between the half-hourly time stamps divided by 30 minutes) for all gridpoints and time stamps. All tendency terms are divided by the dry air mass to obtain more familiar units. The modified code fixes the issue of an unclosed dry theta budget at the lowest gridpoint (see attached figures). The simulations were repeated for km_opt=5.

RELEASE NOTE: To close the dry theta budget at the lowest grid point when use_theta_m is enabled, the surface heat fluxes, and in the case of km_opt=5 also the non-local heat flux, are multiplied with 1+Rv/Rd*qv.

![test_fluxes_flat_thd_3rd+5th](https://user-images.githubusercontent.com/17001470/88155314-389f7380-cc08-11ea-9dac-f7ec6cceeaed.png)
![test_fluxes_flat_thdm_3rd+5th](https://user-images.githubusercontent.com/17001470/88155312-3806dd00-cc08-11ea-8226-4ed46c5c556e.png)
![test_fluxes_flat_fixed-sfc-fluxes_thdm_3rd+5th](https://user-images.githubusercontent.com/17001470/88155315-39380a00-cc08-11ea-8cbd-6b9fd6217c1c.png)
TYPE: bug fix

KEYWORDS: 3D TKE, km_opt, divide by zero

SOURCE: Matthias Göbel (University of Innsbruck)

DESCRIPTION OF CHANGES:
Problem:
When compiling WRF with the -D option, several divide-by-zero errors happen in the km_opt=5 scheme. This PR fixes the problems, and the changes are approved by the developer.

LIST OF MODIFIED FILES:
dyn_em/module_diffusion_em.F

TESTS CONDUCTED:
Jenkins testing is all PASS.

RELEASE NOTE: Fixed divide by zero in the 3D-TKE option km_opt=5.
TYPE: bug fix

KEYWORDS: RRTMG-K option (sw and lw option 14)

SOURCE: Soonyoung Roh and Hwan-Jin Song (National Institute of Meteorological Science, Korea)

DESCRIPTION OF CHANGES:
The bugs include wrong surface downward diagnostic output for long- and short-wave fluxes (clear sky fluxes misplaced in cloudy sky flux arrays), a wrong data statement value for wavenum2 and a wrong value used for bound check for effective size of snow.
Affects radiation driver call arguments for lw and sw scheme and internal numerical values in two places of the shortwave scheme (rrtmg_swk).

This has been confirmed as a correct fix by the developers.

ISSUE:
Fixes wrf-model#1312

LIST OF MODIFIED FILES:
M phys/module_ra_rrtmg_swk.F
M phys/module_radiation_driver.F

TESTS CONDUCTED:
Test reported by source (documented in Issue wrf-model#1312)
Jenkins testing all pass

RELEASE NOTE:
Fixed a bug in surface downward diagnostic output of long- and short-wave fluxes and two other bugs involving wrong numerical values used in the code (Thanks to Roh and Song of NIMS, Korea).
)

TYPE: bug fix

KEYWORDS: tke, isfflx, tke_heat_flux, tke_drag_coefficient, tke_seed

SOURCE: Matthias Göbel (University of Innsbruck)

DESCRIPTION OF CHANGES:
Problem:
The tke seed value is used in the absence of surface drag or a surface heat flux, since there is no way to generate tke without pre-existing tke. So far the usage of the seed value is only tied to the namelist variables tke_drag_coefficient and tke_heat_flux being greater than 0. However, these namelist variables are only applied if no PBL scheme is used, diff_opt=2, and isfflx=0.

If a PBL scheme is used or diff_opt=1, then isfflx=0 is a sufficient condition to get zero fluxes (see also technical notes).

Solution:
Fixed the if statements to enforce a correct setting of the tke seed value that fits to the purpose of the seed.

LIST OF MODIFIED FILES:
dyn_em/module_diffusion_em.F
wrftladj/module_diffusion_em_ad.F
wrftladj/module_diffusion_em_tl.F

TESTS CONDUCTED:
Jenkins testing is all pass.

RELEASE NOTE: Corrected the tke seed value for km_opt 2 and 5 to be non-zero if and only if the surface heat and momentum fluxes are zero depending on isfflx, diff_opt, and bl_pbl_physics (Thanks to Matthias Göbel of University of Innsbruck).
…rf-model#1301)

TYPE: new feature
    
KEYWORDS: raspberry pi
    
SOURCE: Daniel Zurawski and internal
  
DESCRIPTION OF CHANGES:
Additional strings, `armv7l` and `aarch64`, have been added to the first gfortran configuration stanza
so that these may be keywords for search from Raspberry Pi's `uname -a` output. No new
stanza was added for the gfortran target, so no re-numbering is required  for existing scripts that 
are reliant on such things as "32 = Gfortran for Linux with a serial build".

Associated with this PR, wrf-model#1300 (Inconsistent arg types (integer kind) across subroutine call) is
also required to get the Raspberry Pi to build successfully.
 
LIST OF MODIFIED FILES:
modified:   arch/configure.defaults
    
TESTS CONDUCTED:
1. Previously, a Raspberry Pi build returned this:
``` 
*********************************************************
***              ERROR ERROR ERROR ERROR              ***
***                                                   ***
*** Configuration not found in configure.defaults     *** 
*********************************************************
``` 
2. After this PR, the configure command is able to find a target (raspian buster):
``` 
-----------------------------------------------------------------------
Please select from among the following Linux armv7l options:
    
  1. (serial)   2. (smpar)   3. (dmpar)   4. (dm+sm)   GNU (gfortran/gcc)
``` 
3. After this PR, the configure command is able to find a target (ubuntu mate):
```
------------------------------------------------------------------------
Please select from among the following Linux aarch64 options:

  1. (serial)   2. (smpar)   3. (dmpar)   4. (dm+sm)   GNU (gfortran/gcc)
```
4. Jenkins testing is all pass.
    
RELEASE NOTES: Modifications to the WRF code now permit the model to be built on a Raspberry Pi with the GNU/8 compiler.
KEYWORDS: chemistry, emissions, aircraft, indexing

SOURCE: Stacy Walters (formerly NCAR)

DESCRIPTION OF CHANGES:
Problem:
An improper loop indexing that can cause addressing or floating point errors when aircraft emissions are
used.

Solution:
Fix indexing (i - dimension) for the conversion term, from i to its:ite

LIST OF MODIFIED FILES:
M       chem/module_emissions_anthropogenics.F

TESTS CONDUCTED:
1. Jenkins test PASSED

RELEASE NOTE: Improper indexing in a conversion factor when using aircraft emissions for WRF Chem previously caused addressing (seg fault) or floating point errors. This has been corrected by matching the indexing on the LHS with the RHS.
…model#1337)

TYPE: bug fix

KEYWORDS: vertical, interpolation

SOURCE: internal

DESCRIPTION:
The stand-alone vertical interpolation program that is part of the module_initialize_real.F
file allows detailed testing of the existing vertical interpolation process used for ARW
in the real program. It is understood that practically no one except for me ever exercises
this utility.

1. The call to the main vertical interpolation routine, vert_interp, was out of date (missing
the grid id for some diagnostic error prints).

2. A suggestion for a build command is now provided.

3. Locations to use pressure-level or eta-level data are provided.

LIST OF MODIFIED FILES:
M	module_initialize_real.F

TESTS CONDUCTED:
1. The stand-alone code builds and runs properly.
```
 ------------------------------------
 UNIT TEST FOR VERTICAL INTERPOLATION
 ------------------------------------
  
  
 ------------------------------------
 Lagrange Order =            1
 ------------------------------------
  
  
 Level   Pressure     Field
           (Pa)      (generic)
 ------------------------------------
  
 1    102000.000   102000.00    
 2    100000.000   100000.00    
 3     94722.219   94722.219    
 4     89444.445   89444.445    
 5     84166.664   84166.664    
 6     78888.891   78888.891    
 7     73611.109   73611.109    
 8     68333.336   68333.336    
 9     63055.555   63055.555    
10     57777.777   57777.777    
11     52500.000   52500.000    
12     47222.223   47222.223    
13     41944.445   41944.445    
14     36666.668   36666.668    
15     31388.889   31388.889    
16     26111.111   26111.111    
17     20833.334   20833.334    
18     15555.556   15555.556    
19     10277.777   10277.777    
20      5000.000   5000.0000    
  
 Multi-Order Interpolator
 ------------------------------------
  
 Level  Pressure      Field           Field         Field
          (Pa)        Calc            Interp        Diff
 ------------------------------------
  
 1    100327.586    100327.6       100327.6      0.7812500E-02
 2     96982.758    96982.76       96982.76       0.000000    
 3     93637.930    93637.93       93637.93       0.000000    
 4     90293.102    90293.10       90293.10       0.000000    
 5     86948.273    86948.27       86948.27       0.000000    
 6     83603.445    83603.45       83603.45     -0.7812500E-02
 7     80258.617    80258.62       80258.62       0.000000    
 8     76913.797    76913.80       76913.80       0.000000    
 9     73568.969    73568.97       73568.97       0.000000    
10     70224.141    70224.14       70224.14       0.000000    
11     66879.312    66879.31       66879.31       0.000000    
12     63534.484    63534.48       63534.48       0.000000    
13     60189.656    60189.66       60189.66       0.000000    
14     56844.828    56844.83       56844.83       0.000000    
15     53500.000    53500.00       53500.00       0.000000    
16     50155.172    50155.17       50155.17       0.000000    
17     46810.344    46810.34       46810.35     -0.3906250E-02
18     43465.516    43465.52       43465.52       0.000000    
19     40120.688    40120.69       40120.69       0.000000    
20     36775.863    36775.86       36775.87     -0.3906250E-02
21     33431.035    33431.04       33431.04       0.000000    
22     30086.207    30086.21       30086.21       0.000000    
23     26741.379    26741.38       26741.38       0.000000    
24     23396.551    23396.55       23396.55       0.000000    
25     20051.725    20051.72       20051.73     -0.1953125E-02
26     16706.896    16706.90       16706.90       0.000000    
27     13362.068    13362.07       13362.07       0.000000    
28     10017.241    10017.24       10017.24      0.9765625E-03
29      6672.414    6672.414       6672.413      0.4882812E-03
```
2. Due to cpp ifdefs, the modifications have no impact on the traditionally used compilable code.
TYPE: bug fix

KEYWORDS: km_opt=5, 3d tke scheme, use_theta_m=1

SOURCE: internal

DESCRIPTION OF CHANGES:
Problem:
A previous surface flux fix (61029b1) applied a theta_m correction to all levels. However, this should only
be used for qv at the surface.

Solution:
Switch k to kts in one place where this is applied, and pull the invariant assignment out of the k-index DO loop.

LIST OF MODIFIED FILES:
M dyn_em/module_diffusion.F

TESTS CONDUCTED:

Jenkins testing is all pass.
RELEASE NOTE: Addendum to heat-flux theta_m fix (61029b1)* correct pwat function to use dry rho for integration
TYPE: bug fix

KEYWORDS, real, int, integer, wrf_dm_max_int

SOURCE: internal

DESCRIPTION OF CHANGES:
While looking to see if WRF had a function to return the maximum integer from among the
max values of integers on each of the MPI ranks, it was noticed that the return value for the
existing function was REAL. It should be an INTEGER.

There is only a single use of this function in the entire model, in the file `share/module_trajectory.F`.
```
glb_traj_proc(trj) = wrf_dm_max_int( traj_proc(trj) )
```
In that file, the input to the function, `traj_proc`, and the return value, `glb_traj_proc`, are both declared as integers. Therefore, there is no reason for a function to be given a list of integers, and when finding the max value to actually return that value as a real.
```
   integer :: traj_proc(traj_max), glb_traj_proc(traj_max)
```

LIST OF MODIFIED FILES:
M	external/RSL_LITE/module_dm.F

TESTS CONDUCTED:
1. Jenkins tests OK.
2. A small set of trajectories works as expected.
![Screen Shot 2021-01-06 at 6 01 44 PM](https://user-images.githubusercontent.com/12666234/103838081-5f802380-5049-11eb-87ed-648860b7d203.png)
TYPE: text only

KEYWORDS: namelist, readme

SOURCE: internal

DESCRIPTION OF CHANGES:
The trajectory examples.namelist was no longer providing sufficient info to run trajectories.
1. One of the helpful suggestions had a misspelled namelist record. It should have  been `domains`, but it was referred 
to as `domain`.
2. There was no info on the additional options required: `dm_has_traj`.
3. There was no mention of the now mandatory additional text file `wrfinput_traj_d01` that has two namelist records.

LIST OF MODIFIED FILES:
M	test/em_real/examples.namelist

TESTS CONDUCTED:
1. This readme is not part of the build process, or part of compilable code. It will
have no impact of results. Users likely also just skip past the incorrect namelist record
identifier and grab the intended variable (num_traj). All in all, pretty small potatoes.
2. The jenksins tests pass, but no surprise - since we did not change anything that is part of the build.
TYPE: bug fix

KEYWORDS: build, hydro, intel, stanza

SOURCE: Ryan Cabell (NCAR)

DESCRIPTION OF CHANGES:
Problem:
Building coupled WRF/WRF Hydro models fails when choosing configuration INTEL (ifort/icc): HSW/BDW. 

Solution:
Add missing `ifort compiler with icc` text to ARCH line for the HSW/BDW configuration. For completeness, the KNL MIC
configure stanza was also missing the same information, and has been updated.

Neither of these stanzas are involved in the traditional WRF code build, so there is no impact on users of the default WRF
atmosphere model.

ISSUE: For use when this PR closes an issue.
Fixes wrf-model#1325 

LIST OF MODIFIED FILES: 
M       arch/configure.defaults

TESTS CONDUCTED: 
1. Modification causes WRF-Hydro build to complete successfully with the `INTEL (ifort/icc): HSW/BDW` configuration
2. Jenkins testing all positive
…model#1273)

TYPE: Bug fix

KEYWORDS: Fix, obsolescent, non-standard, features

SOURCE: Milan Curcic (University of Miami)

DESCRIPTION OF CHANGES:
Problem:
This PR removes or fixes some obsolescent and non-standard features in several physics modules which were
preventing WRF to be built with IBM XL v16.1.1 on Power9, as reported in wrf-model#1270.

Solution:
1. Replace amax1() occurrences with max()
2. Explicitly cast literal constants that are passed to min() and max() intrinsics so that they are type and kind
compatible with other arguments
3. Replace calls to the non-standard flush() subroutine with the standard flush() statement.

ISSUE:
Fixes wrf-model#1270

LIST OF MODIFIED FILES:
M phys/module_cu_mskf.F
M phys/module_cu_scalesas.F
M phys/module_dust_emis.F
M phys/module_mp_fast_sbm.F
M phys/module_mp_milbrandt2mom.F
M phys/module_mp_p3.F
M phys/module_mp_thompson.F
M phys/module_ra_goddard.F
M phys/module_shcu_deng.F

TESTS CONDUCTED:

These fixes were necessary to allow WRF to be built and run on IBM Power9 with the XL v16.1.1 compiler. The
executables build in both single (./configure) and double (./configure -r8) precision modes.
Jenkins testing is all pass.

RELEASE NOTE: Several physics modules were updated to be more Fortran standard-conforming. This PR removes or fixes some obsolescent and non-standard features in these physics modules which were preventing WRF to be built with IBM XL v16.1.1 on Power9.
twjuliano and others added 8 commits January 12, 2021 15:38
…ent hydrometeors, shortwave irradiances in time series (wrf-model#1284)

TYPE: bug fix, enhancement

KEYWORDS: solar diagnostic, shortwave irradiance, solar zenith, hydrometeor, cloud

SOURCE: Timothy W. Juliano, Ju-Hye Kim, Pedro A. Jimenez, Jared Lee, Thomas Brummet (NCAR/RAL)

DESCRIPTION OF CHANGES:
Bug fixes
Problem:
1. Value of cosine solar zenith angle (coszen) can be slightly <-1 or slightly >1, which results in a nonphysical value for diagnosis of the solar zenith angle
2. Calculating cloud parameters in solar diagnostic package when hydrometeor type is not present

Solution:
1. Bound the values of coszen to be [-1, 1]
2. Add conditional check in solar diagnostic module for presence of hydrometeor type before calculating cloud parameters. Also add check for summation of water path variables to ensure >=0.0 kg/m2.

Enhancement
We have added the Global Horizontal Irradiance (SWDOWN), Direct Normal Irradiance (SWDDNI), and Diffuse Horizontal Irradiance (SWDDIF), as well as the clear-sky Global Horizontal Irradiance (SWDOWNC) and clear-sky Direct Normal Irradiance (SWDDNIC) variables to the tslist when the solar diagnostic package is activated. Furthermore, we add these five variables as calculated by FARMS: SWDOWN2, SWDDNI2, SWDDIF2, SWDOWNC2, and SWDDNIC2.

All 10 variables are output regardless of model physics configuration. Variables SWDOWN, SWDDNI, and SWDDIF are calculated when any shortwave radiation scheme is active. If the RRTMG (ra_sw_physics=4) or RRTMG FAST (ra_sw_physics=24) shortwave radiation scheme or FARMS (swint_opt=2) is activated, then variables SWDOWNC and SWDDNIC are calculated [otherwise, set to MISSING (=-999.0)]. If FARMS (swint_opt=2) is activated, then variables SWDOWN2, SWDDNI2, SWDDIF2, SWDOWNC2, and SWDDNIC2 are calculated [otherwise, set to MISSING (=-999.0)].

LIST OF MODIFIED FILES:
M Registry/Registry.EM_COMMON
M Registry/registry.solar_fields
M phys/module_diag_solar.F
M phys/module_diagnostics_driver.F
M phys/module_radiation_driver.F
M run/README.tslist
M share/wrf_timeseries.F

TESTS CONDUCTED:
1. We have run simulations with and without the fixes. Problem 1: When coszen is within the range [-1, 1], the model results are bit for bit identical before and after the change. Problem 2: The model output confirms that the cloud parameters are not calculated when the hydrometeor type is not present and also that the water path variables are positive. Enhancement: The tslist output is written as expected.
2. Jenkins testing is all PASS.

RELEASE NOTE: Several fixes are introduced for the solar diagnostic package: 1) Correct solar zenith angle calculation (this will have a small effect on radiation for all schemes); and 2) check presence of hydrometeor type before calculating cloud parameters. An enhancement to the package is also introduced: Adding shortwave irradiance variables to the tslist when activating the RRTMG or RRTMG FAST shortwave radiation scheme or FARMS.
…rf-model#1359)

TYPE: bug fix 

KEYWORDS: chemistry, sectional, CCN, mixactivate

SOURCE: Calvin Howes (UCLA), Richard Easter (PNNL), internal

DESCRIPTION OF CHANGES:

Problem:
The sectional scheme diagnostic CCN1, CCN2, ... CCN6 output as computed in mixactivate did not allow for particle 
growth in the computation of volume-mean diameters.  This bug was initially found by Calvin Howes, with the current 
fix proposed by Richard Easter 09/25/2020.  The CCN output fields being diagnostic do not impact actual aerosol 
activation, so cloud properties are not impacted by the bug.

Solution:

Subroutine mixactivate in module_mixactivate.F was modified to use actual time-dependent sizes rather than the default values in computing critical supersaturations and hence CCN.

LIST OF MODIFIED FILES:
M       phys/module_mixactivate.F 

TESTS CONDUCTED: 

1. The first set of attached figures shows CCN6 (CCN at > 1% SS) in the WRF restart files for the lowest model level in a simulation of the Amazon using chem_opt = 11 (RADM2SORG_AQ).  The number of CCN is reduced in areas of cloud formation, suggesting wet removal.  But this preferentially impacts the larger / activated Aitken mode particles, reducing the particle size from the default size.  Thus with the bug fix the diagnosed CCN is reduced relative to the standard code, while the cloud condensate mixing ratio (QCLOUD) is unchanged (second set of slides).

[CCN_bug_fix_test_4.2.2.pptx](https://github.com/wrf-model/WRF/files/5784455/CCN_bug_fix_test_4.2.2.pptx)

2. Jenkins testing is all pass.

RELEASE NOTE: Corrected bug in mixactivate for diagnostic CCN in sectional schemes. Impacts Morrison and Lin et al. microphysics schemes.
…1336)

TYPE: bug fix

KEYWORDS: geopotential, index, horizontal advection, open, specified

SOURCE: Matthias Göbel (University of Innsbruck)

DESCRIPTION OF CHANGES:

When using advection order < 5 (h_sca_adv_order) and open or specified boundary conditions, the upper limits in the horizontal advection loops of geopotential are one too small.

Let's look at the y-advection with advection order = 2, for instance and assume jte=jde. For open and specified BC the lower limit is increased by 1 from jds to jds+1, but the upper limit is decreased by 2 from jde-1 to jtf=jde-1-2=jde-3. It should be jtf=jde-2, however. Starting at line 2092 the jde-1 point is set, but the jde-2 point is not handled anywhere.

The issue is analogous for x-advection and advection order = 4 (jtf=jde-4 instead of jtf=jde-3).

For advection order >= 5, the limits are set correctly, because a different notation is used: jtf = min(jtf,jde-4)

LIST OF MODIFIED FILES: 
dyn_em/module_big_step_utilities_em.F

TESTING CONDUCTED: 
An LES case is used to test this change, with periodic y-boundaries and open x-boundaries, mean x-wind of 10 m/s, h_sca_adv_order=3, and north-south oriented cosine ridge in the center of the domain. Here is the perturbation geopotential after 1 minute integration at k=2:
![ph](https://user-images.githubusercontent.com/17001470/102336394-67d8c780-3f91-11eb-899d-3b05d28c05ab.png)
and here the difference plot:
![ph_diff](https://user-images.githubusercontent.com/17001470/102336506-876ff000-3f91-11eb-9ccf-7615df8adaff.png)

One can clearly see the higher geopotential in the new formulation at i=ide-3. This is the point where the horizontal advection is not carried out by the current algorithm.

After 10 minutes it looks like this:
![ph_600](https://user-images.githubusercontent.com/17001470/102336745-dd449800-3f91-11eb-9b06-94776b70d9c8.png)
![ph_diff_600](https://user-images.githubusercontent.com/17001470/102336749-df0e5b80-3f91-11eb-83ac-1d15a499aae9.png)

It passes all Jenkins tests too.

RELEASE NOTES: When using advection order < 5 (namelist h_sca_adv_order), and either open or specified boundary conditions, the upper limit of the indexing in the horizontal advection loops of geopotential was previously incorrect. For horizontal advection order >= 5 (which is the default), the index limits are correct.
TYPE: bug fix

KEYWORDS: lh_urb2d, diagnostics, noahmp

SOURCE: internal (reported by Xin-Zhong Liang)

DESCRIPTION OF CHANGES:
Problem:
lh_urb2d was inconsistent with other urban budget arrays sh_urb2d, g_urb2d, and rn_urb2d by being the only one multiplied by the urban fraction making it a grid cell average instead of urban area average. This is only a diagnostic for output with using of an urban option together with NoahMP.

Solution:
Divide lh_urb2d by frc_urb2d. Won't affect model run results apart from this diagnostic. These arrays are not in default history output.

LIST OF MODIFIED FILES:
phys/module_sf_noahmpdrv.F

TESTS CONDUCTED:
Only Jenkins tests were performed.
The Jenkins tests are all passing.

RELEASE NOTE: Fix lh_urb2d diagnostic to be urban area average instead of grid cell average consistent with other urb2d diagnostics associated with NoahMP. No effect on run results.
…del#1314)

TYPE: bug fix

KEYWORDS: boundary conditions, symmetric, corners

SOURCE: Matthias Göbel (University of Innsbruck)

DESCRIPTION OF CHANGES:
Problem:
In idealized simulations, the routine `set_physical_bc3d` sets the boundary zone points for periodic, symmetric, and open boundary conditions.
At first, the routine does not set the corner points (e.g., (ids-1,jds-1)): For the y-boundaries, it iterates i only from `MAX(ids,its-1)` to  `MIN(ite+1,ide+istag)` and analogously for the x-boundaries (note that for open BC it actually also tries to set the corner points (see line 896), but not effectively, because the RHS values in lines 903-905 are still 0 there, as the y-BC has not been applied yet).

For doubly-periodic BC, the corner points are set correctly afterward, starting at line 1100. In any other case, however, it seems that the corner points are not assigned and appear as zeros. 

These corner points are later used, e.g. in the SGS UV-flux (titau_21_12) in the diffusion module:
https://github.com/wrf-model/WRF/blob/f311cd5e136631ebf3ebaa02b4b7be3816ed171f/dyn_em/module_diffusion_em.F#L5511-L5517
Thus, because `rho(i_start-1,k,j_start-1)=0` and `xkx(i_start-1,k,j_start-1)=0`, we have `xkxavg(i_start, k, j_start)` much smaller than at the other locations.

This is usually not a problem, since the deformation defor12 and thus also the flux titau_21_12 should be 0 at these corner points when using symmetric boundary conditions on at least one of these boundaries.
I write "should" because that's a theoretical argument. In practice, only the deformation at the ids or jds boundaries is zero. Due to an inconsistency in the symmetric BC for the deformation, the deformation at the ide or jde boundaries is not 0. This problem is described in the following.

The deformation (variable = 'd') is treated like an unstaggered variable: https://github.com/wrf-model/WRF/blob/f311cd5e136631ebf3ebaa02b4b7be3816ed171f/share/module_bc.F#L1003-L1011
Thus, the deformation which was initially zero at the boundary is set to a non-zero value by `dat(i,k,jde) = dat(i,k,jde-1)`.
All staggered variables except the normal velocity (so also the deformation) should be treated equally in the symmetric boundaries since the location of the symmetric boundary is on the staggered grid. Therefore I adjusted the relevant if statement in the second commit.

If open boundary conditions are used on any of the boundaries, only the fluxes on the interior are calculated, so the corner points also don't matter. However, to avoid any current or future side effects due to the missing corner points, corner points are set anyway as it is done in `set_physical_bc2d`.

Solution:

1. commit:
The solution to set the corner points is the same as is already applied in the 2d routine `set_physical_bc2d`.

2. commit:
To treat all variables staggered in a certain direction equally, the variables `jstag` and `istag` are used in the if statement instead of checking the value of `variable`. This was done only for the symmetric BC. For the open BC, it might be necessary, too.

3. commit:
The corner point assignment for doubly-periodic BC is now redundant and was removed.

ISSUE:
Fixes wrf-model#1311

LIST OF MODIFIED FILES: share/module_bc.F

TESTS CONDUCTED:
With the first commit, I tested, whether the corner points for doubly-periodic BC are set in the same way as in the original code. This test allowed removing this redundant part of the code in commit 3.

To compare the different model versions, I ran an idealized simulation with km_opt=2, diff_opt=2, tke_heat_flux=0.2, U = 5 m/s, periodic in x, symmetric in y.
The following figure shows the titau_21_12 flux at k=kds after 20 minutes for three model versions:
left: original, middle: fixed corners (commit 1), right: fixed corners and fixed symmetric BC for the deformation (commit 1 and 2)
![fuy_sgs](https://user-images.githubusercontent.com/17001470/98154601-505be880-1ed5-11eb-9f0a-6eef1175e730.png)

All versions have titau_21_12=0 at j=jds. In the left and middle panel, the fluxes at jde and jde-1 are almost identical. However, the original version has fluxes with lower magnitudes at the upper left and right corners (as the corners of xkmh and rho are not set as explained above). In the right panel, the flux at j=jde is now also zero as it should be.

The following figure shows a time series of U at the upper left corner:
![u_timeseries](https://user-images.githubusercontent.com/17001470/98154606-52be4280-1ed5-11eb-93d3-563062274e5c.png)

We can see that fixing the symmetric BC to have zero fluxes at j=jde does have a significant effect.
Fixing the corners (commit 1) only has a minor effect, which is maximized at the corner points (j=jde, i=ids, and ide) as can be seen here:
![u_diff](https://user-images.githubusercontent.com/17001470/98154624-55b93300-1ed5-11eb-906d-b8aaef7f1596.png)

RELEASE NOTE: 
Missing corner points are set when setting boundary conditions of 3D variables and all staggered variables are treated equally in symmetric BC. Thereby the periodic BC become cleaner and more consistent with set_physical_bc2d and the symmetric BC are fixed as horizontal deformation is treated now as a staggered variable which leads to correct BC for SGS momentum fluxes at the northern boundaries.
Type: Text only
Keywords: prepare for release of WRFV4.2.2
Source: internal
Description of Changes: modify README and inc/version_decl
List of modified files:
M README
M inc/version_decl
Test conducted: not necessary
Finalize WRFV4.2.2 by merging bug fixes from release-v4.2.2 branch onto master.
See the annotation of the v4.2.2 tag for full release notes (git show v4.2.2)
@letmaik
Copy link
Author

letmaik commented Jan 17, 2021

rel_err_ext_boxplot(1)
rel_err_ext_boxplot

@letmaik letmaik marked this pull request as ready for review January 17, 2021 15:37
@letmaik letmaik requested a review from dmey January 17, 2021 15:37
Copy link

@dmey dmey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The REAMDE.md still has badges for Travis and Appveyor -- can you remove them now or I can do it directly into the main branch (wrf-cmake). Rest LGTM!

@letmaik letmaik merged commit b6657ce into wrf-cmake Jan 17, 2021
@letmaik letmaik deleted the letmaik/wrf-4.2.2 branch January 17, 2021 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet