GMT installation clashes with SHTOOLS

[konwill]

Hi,
I very much appriciate your efforts and would welcome any further developments in this direction. When trying gmt python I followed the guidelines available and could install it without a problem. However, a colleague could not run his scripts any more since gmt python changed some packages, it appears. Judging from error messages it traces back to the fftw part that is used by both gmt and shtools.

Best,
Konwill

[leouieda]

Hi @konwill thanks for reporting this!

Could you post the actual error messages?

I can see from the SHTOOLS docs that they recommend installing the system packages for fftw. We use the conda packages so that we can easily distribute the GMT binaries as well. I can see how having two versions of fftw can mess things up. Unfortunately, we can't use system libraries in a way that makes GMT portable or easily installable.

One recommendation I would have is to install GMT and GMT/Python in a separate conda environment. See this comment.

The GMT/Python wrapper doesn't require the conda version of GMT. You could also build GMT from source (you'll need the trunk version of the code from the repository). This way, GMT would be linked to the system fftw and wouldn't install a separate version.

[leouieda]

The GMT/Python wrapper doesn't require the conda version of GMT. You could also build GMT from source (you'll need the trunk version of the code from the repository). This way, GMT would be linked to the system fftw and wouldn't install a separate version.

This is a way that people could install GMT/Python along side an existing SHTOOLS install.

Another option would be to build conda packages for SHTOOLS. This way, they will be linked to the same libraries as GMT. It shouldn't be too hard and I would gladly help to set this up.

[MarkWieczorek]

I've just added a building a conda package for SHTOOLS as a milestone.... https://github.com/SHTOOLS/SHTOOLS/milestone/12

Of course we would welcome the help doing this! @MMesch started to look into this awhile ago, but I don't think any real progress was made.

[konwill]

Hi again,
thanks for your suggestions regarding the installation. I might give it a go once I find the time for it.
As for the error message I only have the info below from my colleague.

Traceback (most recent call last):
  File "./00_load_data.py", line 18, in <module>
    import pyshtools
  File ".../python3.6/site-packages/pyshtools/__init__.py", line 47, in <module>
    from . import shtools
  File ".../python3.6/site-packages/pyshtools/shtools/__init__.py", line 15, in <module>
    from .._SHTOOLS import PlmBar
ImportError: .../python3.6/site-packages/pyshtools/_SHTOOLS.cpython-36m-x86_64-linux-gnu.so: undefined symbol: dfftw_destroy_plan_

Thanks for looking after this issue.

[MarkWieczorek]

One potential problem is that FFTW for shtools needs to be built with fortran bindings. Its my understanding that a "default" build won't do this, and that you need to specify this explicitly. On OSX using brew, this would be installed using

brew install fftw --with-fortran

Not sure how the conda package deals with this.

[leouieda]

One potential problem is that FFTW for shtools needs to be built with fortran bindings.

Bingo! That seems to be the problem with the conda-forge FFTW: https://github.com/conda-forge/fftw-feedstock/blob/master/recipe/build.sh#L6

This will be a problem for the conda package for SHTOOLS. A difficulty in getting conda packages is that every dependency needs to have a package so that you can have an isolated and reproducible environment. So we would need to have a SHTOOLS compatible FFTW first.

There is already an issue on the feedstock (conda-forge/fftw-feedstock#16) but it's been sitting there for almost a year. I might take a shot at implementing this when I have some time.

And of course I'm more than happy to help setup the conda package!

[seisman]

Can this issue be closed? Seems conda-forge FFTW already enables Fortran bindings (conda-forge/fftw-feedstock#16, conda-forge/fftw-feedstock#49).

[MarkWieczorek]

Thanks for letting us know!