GERun is a simple, extensible wrapper for different versions of mpirun such that users don't need to know the implementation specfic options when running MPI programs on Legion.
The code is licensed under a 4-clause, BSD-style license, see LICENSE.txt for details.
GERun replaces mpirun with a single "gerun" command and autofills in options based on the GE environment.
The intitial version is described in this blog-post:
This version is the public, tidied up version of the code.
GERun allows users to replace complicated formulations like:
mpirun -m $TMPDIR/machines -np $NSLOTS $HOME/src/madscience/mad
or
mpirun --rsh=ssh -machinefile $TMPDIR/machines -np $NSLOTS $HOME/src/madscience/mad
depending on their MPI implementation with:
gerun $HOME/src/madscience/mad
GERun version iii supports mixed mode codes in some MPI environments (Intel MPI, OpenMPI without SGE integration) by running $NSLOTS/$OMP_NUM_THREADS MPI processes, and provides a tool "ppn" which calculates the number of processors per node. This means that on the new Research Computing OS software stack you can portably (over the node types in the cluster) run with OpenMP threads within a node, MPI between a node should you desire by doing this:
export OMP_NUM_THREADS=`ppn`
gerun $HOME/src/madscience/mad
Alternatively, if you always want to run 2 threads per MPI task you can do this:
export OMP_NUM_THREADS=2
gerun $HOME/src/madscience/mad
GERun will then run $NSLOTS/2 MPI processes with 2 OpenMP threads each.
On Legion we use environment modules and so the instructions will be written assuming this.
-
Unpack the code somewhere.
-
Add the path to the code into $PATH (on Legion this is done as part of the "sge" module), and set the $GERUN_PATH to the same location.
-
Inside the module for each MPI implementation set $GERUN_LAUNCHER to the appropriate version for that MPI.
Provided with the code are:
MPI | $GERUN_LAUNCHER | Wrapper file |
---|---|---|
Qlogic | qlc | gerun-qlc |
Intel | intel | gerun-intel |
OpenMPI | openmpi | gerun-openmpi |
OpenMPI with SGE | openmpi-sge | genrun-openmpi-sge |
To add a new MPI implementation, you simply need to create a new gerun- wrapper and set $GERUN_LAUNCHER appropriately.