Initialise PETSc, drop LinSolveExprs in lower_petsc, add PETScStruct, create matvec callback and move files into PETSc folder #14
Conversation
devito/types/petsc.py
Outdated
| @@ -247,3 +247,24 @@ def solver_parameters(self): | |||
| return self._solver_parameters | |||
|
|
|||
| func = Reconstructable._rebuild | |||
|
|
|||
|
|
|||
| class PETScStruct(CompositeObject): | |||
There was a problem hiding this comment.
placing this question here, but I should have asked time ago. Do we actually need types/petsc?
I'm now thinking PETSc support might deserve its own folder in the root directory, alongside ir, types, passes, mpi, etc
I'd prefer it centralised rather than distributed, but perhaps it's just me, thoughts @ZoeLeibowitz @mloubout @EdCaunt
devito/petsc/iet/passes.py
Outdated
| dm_get_app_context = Call(petsc_call, [Call('DMGetApplicationContext', arguments=[ | ||
| objs['da'], Byref(struct._C_symbol)])]) | ||
|
|
||
| dm_get_local_xvec = Call(petsc_call, [Call('DMGetLocalVector', arguments=[ |
There was a problem hiding this comment.
I agree to improve readability you may add simple utility functions creating Call(... [Call(....)])
devito/petsc/iet/passes.py
Outdated
|
|
||
| func_return = Call('PetscFunctionReturn', arguments=[0]) | ||
|
|
||
| matvec_body = List(body=[ |
There was a problem hiding this comment.
yes, that or one word per line, it doesn't matter if it gets long
devito/petsc/iet/passes.py
Outdated
| unique_targets = list(set([i.expr.rhs.target for i in petsc_nodes])) | ||
| init = init_petsc(**kwargs) | ||
| struct = build_petsc_struct(iet) | ||
| objs = build_core_objects(unique_targets[-1], **kwargs) |
There was a problem hiding this comment.
This seems unsafe? list(set()) returns an iterable with non-deterministic order, so taking the last item doesn't guarantee getting the same thing every time
There was a problem hiding this comment.
You might want to sort here for determinism.
There was a problem hiding this comment.
I think this is ok, at least for now, because I assume that all targets have the same Grid and so it does not matter which one I use
| @@ -1169,8 +1140,9 @@ class Callback(Call): | |||
| engine fails to bind the callback to a specific Call. Consequently, | |||
| errors occur during the creation of the call graph. | |||
| """ | |||
|
|
|||
| def __init__(self, name, retval, param_types): | |||
| # TODO: Create a common base class for Call and Callback to avoid | |||
There was a problem hiding this comment.
looks like an easy one
devito/petsc/types.py
Outdated
| @cached_property | ||
| def _shape_with_inhalo(self): | ||
| """ | ||
| Shape of the domain+inhalo region. The inhalo region comprises the | ||
| outhalo as well as any additional "ghost" layers for MPI halo | ||
| exchanges. Data in the inhalo region are exchanged when running | ||
| Operators to maintain consistent values as in sequential runs. | ||
|
|
||
| Notes | ||
| ----- | ||
| Typically, this property won't be used in user code, but it may come | ||
| in handy for testing or debugging | ||
| """ | ||
| return tuple(j + i + k for i, (j, k) in zip(self.shape, self._halo)) | ||
|
|
||
| @cached_property | ||
| def shape_allocated(self): | ||
| """ | ||
| Shape of the allocated data of the Function type object from which | ||
| this PETScArray was derived. It includes the domain and inhalo regions, | ||
| as well as any additional padding surrounding the halo. | ||
|
|
||
| Notes | ||
| ----- | ||
| In an MPI context, this is the *local* with_halo region shape. | ||
| """ | ||
| return DimensionTuple(*[j + i + k for i, (j, k) in zip(self._shape_with_inhalo, |
There was a problem hiding this comment.
This looks like a duplication to what devito has already?
docker/Dockerfile.devito
Outdated
| apt-get update && apt-get install -y libopenblas-serial-dev git pkgconf mpich && \ | ||
| python3 -m venv /venv && \ | ||
| /venv/bin/pip install --no-cache-dir --upgrade pip && \ | ||
| /venv/bin/pip install --no-cache-dir --no-binary numpy numpy && \ |
There was a problem hiding this comment.
It is to avoid using pre-built binary wheels for numpy. This is important to ensure that PETSc and numpy are built against the same BLAS/LAPACK library. This is why the PETSc install stage is before Devito.
c0751dc
to
e7e29d3
Compare
No description provided.