Notes from parallel BMI planning meeting II

[mdpiper]

On 2023 Nov 16, we held a second meeting (over Zoom) to discuss how BMI could work in a parallel environment. In attendance were @hrajagers, @jdhughes-usgs, @rcabell, @scrasmussen, @mcflugen, and @mdpiper. (These are Mark's sketchy notes.)

Since the last meeting, Bert put forth his suggestions for a parallel BMI specification in the hrajagers/parallel-bmi branch (see #132 (comment)). Ryan and Soren had a chance to examine it.

Some initial feedback from Soren:

• Change "MPI threads" to "MPI ranks" in docs.
• Global grid size viaget_grid_partition_size or get_grid_size? Group recommendation: don't change the behavior of the existing function.
• Support older int and newer MPI_Comm types in the parallel_initialize function. It should be generic to handle both.
• Minor comments on MPI style in parallel_initialize.

Bert gave a little more context on why he chose parallel_initialize instead of, e.g., a set_communicator function; the former suggests the possibility of more actions in a single step to set up a parallel environment.
Eric has a preference for the latter so as not to confuse initialize.
Ryan: Should we specify the order in which these functions are called?
Bert: Call set_communicator before initialize.
Bert: How do we keep calls to serial and parallel models compatible in a framework? How can tell which is being run?
Eric: We'll need to figure this out for all extensions. We'll need a get_extensions function.
Bert: Call get_extensions before initialize.

Soren: Currently working with MPI_COMM_WORLD and subcommunicators in WRF-Hydro, looking to talk with MODFLOW.
Joe: There's some information on using communicators in the MODFLOW XMI.

Eric: We should have a parallel namespace prefix for this set of extensions. It could be simple, likeget_var_parallel_* and get_grid_parallel_*.
Bert: Could this open the door to many repeated functions between serial and parallel?
Group: We should have more discussion of how the new functions are named.

MP: I didn't capture the following discussion well.
Eric: Framework would have responsibility for figuring out which model is serial and which is parallel.
Bert: get_value returns size and content for the part of model on that MPI rank.
Soren: Identify where data are being collected from--which MPI rank.

Soren: Current work on WRF-Hydro and MODFLOW assumes both models are partitioned the same. If they differ, that will be harder & he'll have to think about it.
Bert: In his work, partitions in coupled models are matched through ESMF or OASIS, usually as a pre step.
Soren: So, regridding through ESMF or OASIS?
Eric: Yes, user will be responsible.

Bert: Discuss comments from today on forum.
Bert: We need extension framework for BMI. Becoming more critical.
Joe: XMI has similar functions which could inform the naming of the parallel BMI functions.
Joe: Need for extension framework.

Soren: Let's meet again in about two months.