-
Notifications
You must be signed in to change notification settings - Fork 115
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
build: refactored meson files for cray build (#1258)
* Refactor meson files to build a CRAY parallel version of MODFLOW using cray-mpich (7.7.19) and cray-petsc (3.14.5) * Bumped minimum version of meson to 1.1.0. * Updated `PARALLEL.md` with information on installing OpenMPI and PETSc on MacOS using Homebrew. * Added license for PETSc (this is only added to the license when MODFLOW is built with PETSc) * Added functionality to use MODFLOW 6 PROFESSIONAL in the header written to the screen and listing files when MODFLOW 6 is built with PETSC * Added minimal MODFLOW simulation files in `.mf6minsim` directory that can be used for testing serial and parallel builds * Update makefiles
- Loading branch information
1 parent
705b2b2
commit b0c158c
Showing
39 changed files
with
560 additions
and
127 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# Building MODFLOW 6 on HPC systems | ||
|
||
## SLURM job | ||
|
||
``` | ||
sbatch --reservation=dev cray-meson-build.slurm.batch | ||
``` | ||
|
||
## Interactive job | ||
|
||
``` | ||
module switch PrgEnv-${PE_ENV,,} PrgEnv-intel | ||
module load cray-petsc meson ninja | ||
export PKG_CONFIG_PATH=/opt/cray/pe/mpt/7.7.19/gni/mpich-intel/16.0/lib/pkgconfig:/opt/cray/pe/petsc/3.14.5.0/real/INTEL/19.1/x86_skylake/lib/pkgconfig:$PKG_CONFIG_PATH | ||
srun --reservation=dev --account=impd --pty bash | ||
meson setup builddir -Ddebug=false --prefix=$(pwd) --libdir=bin -Dcray=true -Ddebug=false --wipe | ||
meson install -C builddir | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
#!/bin/bash | ||
|
||
#SBATCH --job-name=meson-MODFLOW-build | ||
#SBATCH --nodes=1 | ||
#SBATCH --ntasks=2 | ||
#SBATCH --account=impd | ||
#SBATCH --time=00:10:00 | ||
#SBATCH --output=slurm-%j.out | ||
|
||
# load appropriate modules | ||
module switch PrgEnv-${PE_ENV,,} PrgEnv-intel | ||
module load cray-petsc meson ninja | ||
export PKG_CONFIG_PATH=/opt/cray/pe/mpt/7.7.19/gni/mpich-intel/16.0/lib/pkgconfig:/opt/cray/pe/petsc/3.14.5.0/real/INTEL/19.1/x86_skylake/lib/pkgconfig:$PKG_CONFIG_PATH | ||
|
||
# list loaded modules | ||
module list | ||
|
||
# move to root directory | ||
cd .. | ||
|
||
# define paths relative to the root directory | ||
MODFLOW6ROOT=$(pwd) | ||
BINDIR=$MODFLOW6ROOT/bin | ||
TESTDIR=$MODFLOW6ROOT/.mf6minsim | ||
|
||
# build MODFLOW 6 | ||
CC=cc CXX=CC F77=ftn F90=ftn FC=ftn meson setup builddir --prefix=$(pwd) --libdir=bin -Dcray=true -Ddebug=false --wipe | ||
meson install -C builddir | ||
|
||
# test MODFLOW 6 build | ||
cd $TESTDIR | ||
|
||
# serial run | ||
$BINDIR/mf6 | ||
|
||
# parallel run | ||
srun $BINDIR/mf6 -p |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
-ksp_type cg | ||
-pc_type bjacobi | ||
-sub_pc_type ilu | ||
-sub_pc_factor_levels 2 | ||
-dvclose 1.0e-6 | ||
-options_left no |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
# File generated by Flopy version 3.3.7 on 06/01/2023 at 12:15:38. | ||
BEGIN options | ||
END options | ||
|
||
BEGIN dimensions | ||
MAXBOUND 1 | ||
END dimensions | ||
|
||
BEGIN period 1 | ||
1 1 1 1.00000000 | ||
END period 1 | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
# File generated by Flopy version 3.3.7 on 06/01/2023 at 12:15:38. | ||
BEGIN options | ||
END options | ||
|
||
BEGIN dimensions | ||
NLAY 1 | ||
NROW 1 | ||
NCOL 5 | ||
END dimensions | ||
|
||
BEGIN griddata | ||
delr | ||
CONSTANT 100.00000000 | ||
delc | ||
CONSTANT 100.00000000 | ||
top | ||
CONSTANT 10.00000000 | ||
botm | ||
CONSTANT -100.00000000 | ||
END griddata | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
# File generated by Flopy version 3.3.7 on 06/01/2023 at 12:15:38. | ||
BEGIN griddata | ||
strt | ||
CONSTANT 0.00000000 | ||
END griddata | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
# File generated by Flopy version 3.3.7 on 06/01/2023 at 12:15:38. | ||
BEGIN options | ||
END options | ||
|
||
BEGIN packages | ||
DIS6 leftmodel.dis dis | ||
IC6 leftmodel.ic ic | ||
NPF6 leftmodel.npf npf | ||
CHD6 leftmodel.chd chd_0 | ||
END packages | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
# File generated by Flopy version 3.3.7 on 06/01/2023 at 12:15:38. | ||
BEGIN options | ||
SAVE_FLOWS | ||
SAVE_SPECIFIC_DISCHARGE | ||
END options | ||
|
||
BEGIN griddata | ||
icelltype | ||
CONSTANT 1 | ||
k | ||
CONSTANT 1.00000000 | ||
END griddata | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
# File generated by Flopy version 3.3.7 on 06/01/2023 at 12:15:38. | ||
BEGIN options | ||
END options | ||
|
||
BEGIN timing | ||
TDIS6 par_gwf01-1d.tdis | ||
END timing | ||
|
||
BEGIN models | ||
gwf6 leftmodel.nam leftmodel | ||
gwf6 rightmodel.nam rightmodel | ||
END models | ||
|
||
BEGIN exchanges | ||
GWF6-GWF6 par_gwf01-1d.gwfgwf leftmodel rightmodel | ||
END exchanges | ||
|
||
BEGIN solutiongroup 1 | ||
ims6 par_gwf01-1d.ims leftmodel rightmodel | ||
END solutiongroup 1 | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
# File generated by Flopy version 3.3.7 on 06/01/2023 at 12:15:38. | ||
BEGIN options | ||
auxiliary ANGLDEGX CDIST | ||
END options | ||
|
||
BEGIN dimensions | ||
NEXG 1 | ||
END dimensions | ||
|
||
BEGIN exchangedata | ||
1 1 5 1 1 1 1 50.00000000 50.00000000 100.00000000 0.00000000 100.00000000 | ||
END exchangedata | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# File generated by Flopy version 3.3.7 on 06/01/2023 at 12:15:38. | ||
BEGIN options | ||
PRINT_OPTION all | ||
END options | ||
|
||
BEGIN nonlinear | ||
OUTER_DVCLOSE 1.00000000E-08 | ||
OUTER_MAXIMUM 100 | ||
UNDER_RELAXATION dbd | ||
END nonlinear | ||
|
||
BEGIN linear | ||
INNER_MAXIMUM 300 | ||
INNER_DVCLOSE 1.00000000E-08 | ||
inner_rclose 0.00100000 | ||
LINEAR_ACCELERATION cg | ||
RELAXATION_FACTOR 0.0000000 | ||
preconditioner_levels 2 | ||
END linear | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
# File generated by Flopy version 3.3.7 on 06/01/2023 at 12:15:38. | ||
BEGIN options | ||
TIME_UNITS days | ||
END options | ||
|
||
BEGIN dimensions | ||
NPER 1 | ||
END dimensions | ||
|
||
BEGIN perioddata | ||
1.00000000 1 1.00000000 | ||
END perioddata | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
# File generated by Flopy version 3.3.7 on 06/01/2023 at 12:15:38. | ||
BEGIN options | ||
END options | ||
|
||
BEGIN dimensions | ||
MAXBOUND 1 | ||
END dimensions | ||
|
||
BEGIN period 1 | ||
1 1 5 10.00000000 | ||
END period 1 | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
# File generated by Flopy version 3.3.7 on 06/01/2023 at 12:15:38. | ||
BEGIN options | ||
XORIGIN 500.00000000 | ||
YORIGIN 0.00000000 | ||
END options | ||
|
||
BEGIN dimensions | ||
NLAY 1 | ||
NROW 1 | ||
NCOL 5 | ||
END dimensions | ||
|
||
BEGIN griddata | ||
delr | ||
CONSTANT 100.00000000 | ||
delc | ||
CONSTANT 100.00000000 | ||
top | ||
CONSTANT 10.00000000 | ||
botm | ||
CONSTANT -100.00000000 | ||
END griddata | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
# File generated by Flopy version 3.3.7 on 06/01/2023 at 12:15:38. | ||
BEGIN griddata | ||
strt | ||
CONSTANT 0.00000000 | ||
END griddata | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
# File generated by Flopy version 3.3.7 on 06/01/2023 at 12:15:38. | ||
BEGIN options | ||
END options | ||
|
||
BEGIN packages | ||
DIS6 rightmodel.dis dis | ||
IC6 rightmodel.ic ic | ||
NPF6 rightmodel.npf npf | ||
CHD6 rightmodel.chd chd_0 | ||
END packages | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
# File generated by Flopy version 3.3.7 on 06/01/2023 at 12:15:38. | ||
BEGIN options | ||
SAVE_FLOWS | ||
SAVE_SPECIFIC_DISCHARGE | ||
END options | ||
|
||
BEGIN griddata | ||
icelltype | ||
CONSTANT 1 | ||
k | ||
CONSTANT 1.00000000 | ||
END griddata | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.