MPI code using sub-optimal communication patterns

[donaldwj]

Many places in the existing MPI communication code are using sub-optimal communication patterns. For example uses send and recv in a loop to achieve the results of using mpi_scatter, mpi_gather, mpi_reduce, or mpi_allreduce.

Expected Behavior

Where possible high level communications functions should be used instead of send and recv based logic

Current Behavior

Mpi_send and mpi_recv are being used in places where better communication functions exist.

Possible Solution

Change communication functions to use high level mpi functions when possible