forked from ufs-community/ufs-weather-model
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add support for Gaea C5 (includes PR ufs-community#1977: new ccpp SDF…
…s added to support RRFS multiphysics ensemble and add tob in ocean output)(Includes PR ufs-community#1997) (ufs-community#1784) * Reinstate all the c5 changes that I botched with my git ignorance. * Update ufs_common. Upate c5 modulefile. * TPN adjustments for C5 regional_atmaq tests. * Remove set -eu from module-setup.sh * Update rt.sh adding new DISKNM for Gaea_C5 * updates for gaea-c5 modulefiles, scripts * Update rt.conf fix gaea machine name
- Loading branch information
1 parent
7407bcf
commit f42e347
Showing
29 changed files
with
9,706 additions
and
4,464 deletions.
There are no files selected for viewing
Submodule FV3
updated
6 files
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| set(PARALLEL_NETCDF ON CACHE BOOL "Enable parallel NetCDF" FORCE) | ||
| set(MOM6_Extra_FORTRAN_FLAGS "-xsse2") | ||
| set(HYCOM_Extra_FORTRAN_FLAGS "-xSSE4.2") | ||
| set(HYCOM_Extra_C_FLAGS "-xSSE4.2") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| help([[ | ||
| This module loads libraries required for building and running UFS Weather Model | ||
| on the NOAA RDHPC machine Gaea C5 using Intel-2023.1.0. | ||
| ]]) | ||
|
|
||
| whatis([===[Loads libraries needed for building the UFS Weather Model on Gaea C5 ]===]) | ||
|
|
||
| load("PrgEnv-intel/8.3.3") | ||
| load("intel-classic/2023.1.0") | ||
| load("cray-mpich/8.1.25") | ||
| load("python/3.9.12") | ||
|
|
||
| prepend_path("MODULEPATH", "/lustre/f2/dev/wpo/role.epic/contrib/spack-stack/c5/spack-stack-1.5.0/envs/unified-env/install/modulefiles/Core") | ||
| prepend_path("MODULEPATH", "/lustre/f2/dev/wpo/role.epic/contrib/spack-stack/c5/modulefiles") | ||
|
|
||
| stack_intel_ver=os.getenv("stack_intel_ver") or "2023.1.0" | ||
| load(pathJoin("stack-intel", stack_intel_ver)) | ||
|
|
||
| stack_cray_mpich_ver=os.getenv("stack_cray_mpich_ver") or "8.1.25" | ||
| load(pathJoin("stack-cray-mpich", stack_cray_mpich_ver)) | ||
|
|
||
| stack_python_ver=os.getenv("stack_python_ver") or "3.10.8" | ||
| load(pathJoin("stack-python", stack_python_ver)) | ||
|
|
||
| load("ufs_common") | ||
| load("nccmp/1.9.0.1") | ||
|
|
||
| unload("darshan-runtime") | ||
| unload("cray-libsci") | ||
|
|
||
| setenv("CC","cc") | ||
| setenv("CXX","CC") | ||
| setenv("FC","ftn") | ||
| setenv("CMAKE_Platform","gaea-c5.intel") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| #!/bin/bash -l | ||
| #SBATCH -e err | ||
| #SBATCH -o out | ||
| #SBATCH --account=@[ACCNR] | ||
| ##SBATCH --qos=@[QUEUE] | ||
| #SBATCH --clusters=es | ||
| #SBATCH --partition=eslogin_c5 | ||
| #SBATCH --nodes=1 | ||
| #SBATCH --ntasks-per-node=8 | ||
| #SBATCH --time=180 | ||
| #SBATCH --job-name="@[JBNME]" | ||
|
|
||
| set -eux | ||
|
|
||
| echo -n " $( date +%s )," > job_timestamp.txt | ||
| echo "Compile started: " `date` | ||
|
|
||
| @[PATHRT]/compile.sh @[MACHINE_ID] "@[MAKE_OPT]" @[COMPILE_NR] @[RT_COMPILER] | ||
|
|
||
| echo "Compile ended: " `date` | ||
| echo -n " $( date +%s )," >> job_timestamp.txt |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| #!/bin/bash -l | ||
| #SBATCH -e err | ||
| #SBATCH -o out | ||
| #SBATCH --job-name="@[JBNME]" | ||
| #SBATCH --account=@[ACCNR] | ||
| #SBATCH --qos=@[QUEUE] | ||
| #SBATCH --clusters=c5 | ||
| #SBATCH --partition=batch | ||
| #SBATCH --nodes=@[NODES] | ||
| #SBATCH --ntasks-per-node=@[TPN] | ||
| #SBATCH --time=@[WLCLK] | ||
|
|
||
| set -eux | ||
| echo -n " $( date +%s )," > job_timestamp.txt | ||
|
|
||
| set +x | ||
| MACHINE_ID=gaea-c5 | ||
| source ./module-setup.sh | ||
| module use --prepend $PWD/modulefiles | ||
| module load modules.fv3 | ||
| module list | ||
| set -x | ||
|
|
||
| echo "Model started: " `date` | ||
|
|
||
| export OMP_NUM_THREADS=@[THRD] | ||
| export OMP_STACKSIZE=1024M | ||
| export NC_BLKSZ=1M | ||
| export ESMF_RUNTIME_PROFILE=ON | ||
| export ESMF_RUNTIME_PROFILE_OUTPUT="SUMMARY" | ||
|
|
||
| # Avoid job errors because of filesystem synchronization delays | ||
| sync && sleep 1 | ||
|
|
||
| srun --label -n @[TASKS] ./fv3.exe | ||
|
|
||
| echo "Model ended: " `date` | ||
| echo -n " $( date +%s )," >> job_timestamp.txt |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.