New version of GtoL, using MPI Datatypes.

[dmitrypek]

…Can be used to send individual levels or all levels at once.

[FussyDuck]

All committers have signed the CLA.

[samhatfield]

On ECMWF HPC at least, I need to add MPI as a required package for the MPI CMake feature, and MPI::MPI_Fortran as a private library on the trans_${prec} target. Otherwise compilation of trgtol_mod.F90 fails because mpi.mod is not on the include path (USE MPI has been added to this file).

@dmitrypek could you make these changes and push to your branch?

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 89c513c..f66ff1d 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -20,7 +20,8 @@ ecbuild_find_package( NAME fiat REQUIRED )

 ecbuild_add_option( FEATURE MPI
                     DESCRIPTION "Support for MPI distributed memory parallelism"
-                    CONDITION fiat_HAVE_MPI )
+                    CONDITION fiat_HAVE_MPI
+                    REQUIRED_PACKAGES "MPI COMPONENTS Fortran")

 ecbuild_add_option( FEATURE OMP
                     DEFAULT ON

diff --git a/src/trans/CMakeLists.txt b/src/trans/CMakeLists.txt
index 256ad41..b6592b3 100644
--- a/src/trans/CMakeLists.txt
+++ b/src/trans/CMakeLists.txt
@@ -59,6 +59,7 @@ foreach( prec sp dp )
                        $<INSTALL_INTERFACE:include/ectrans>
                        $<INSTALL_INTERFACE:include>
       PUBLIC_LIBS      fiat parkind_${prec}
+      PRIVATE_LIBS     MPI::MPI_Fortran
     )
     ectrans_target_fortran_module_directory(
       TARGET            trans_${prec}

Note that, for various reasons, we prefer not to use the MPI API directly from code in the IFS and dependencies, including ecTrans. Instead we prefer to use our own wrapper on top of MPI, MPL (part of the FIAT library). This is fine for now, but in future we should find a way to replace the MPI_ calls with equivalent MPL_ calls.

[samhatfield]

You should be able to git apply this file directly:

diff.patch

[samhatfield]

I built and ran a test, but it crashed with a segfault. The error suggests the problem is the IND variable of TRGTOL_COMM. This variable is referenced here before it's initialised. Looks like the commented-out code above initialises it. Could you look into this @dmitrypek ?

[dmitrypek]

This problem should be fixed now. Please give it a try.

[samhatfield]

Thanks Dmitry - I was able to build and test it. By the way, there's a typo on line 778 - SEND_BY_LEVELS.

[dmitrypek]

Thanks - typo fixed. Let me know if you have any questions.

[samhatfield]

Thanks @dmitrypek. I'm doing some benchmarks so we can discuss this in our next meeting.