Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It is time to make GDAL a required prerequisite #6258

Merged
merged 8 commits into from
Jan 26, 2022
Merged

Conversation

PaulWessel
Copy link
Member

Lots of GMT functionality now depends on GDAL, from handling image i/o, including multilevel geotiff data files, making geotiffs from plots, reading a variety of grid formats as well as handling shapefiles (there are plans to handle more table formats via the GDAL/OGR interface). We also plan to start using more proj features as available via GDAL (we will eventually also make PROJ a requirement but since PROJ is required by GDAL it does not really add anything extra). This PR makes the practicality of depending on GDAL (already included in all our installers) formal.

I have left it at WIP for now to make sure we first all agree that

  1. Yes, this needs to be done. Having > 60 #ifdef HAVE_GDAL around the code is complicating things and those 13 people who manage to use GMT but cannot handle the requirement of GDAL simply need to get onboard for the sake of the rest of us.
  2. We have updated any documentation in this repository that mention GDAL as an optional prerequisite.
  3. We have let the CI and the various developers had a change to test this branch and find it satisfactory.
  4. We have ensured our CMake settings are correct so that Linux distributions will pick up on the change once 6.4.0 gets released later this spring.

Once we approve and merge I believe there are other repos (website, others?) that will need some updates. Let us know by adding comments herein.

Lots of GMT functionailty depends on GDAL, from handing images to odd grid formats to shapefiles, and there are plans to handle more table formats via ogr.  This PR makes the practicailty of depending on GDAL (already included in the installers) formal.
@PaulWessel PaulWessel added the maintenance Boring but important stuff for the core devs label Jan 23, 2022
@PaulWessel PaulWessel added this to the 6.4.0 milestone Jan 23, 2022
@PaulWessel PaulWessel self-assigned this Jan 23, 2022
Copy link
Member

@leouieda leouieda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Completely sensible thing to do 👍

If I remember correctly, we already have it as a required dependency on the conda-forge package from the start.

@anbj
Copy link
Contributor

anbj commented Jan 24, 2022

This is good. gdal is good. That said, is this primarily because of gdal libs (i.e. code) being used in gmt, or the drive to make things easier for the user? Having several tools in a toolbox to do different things is by definition (almost) always a good thing. Please do not create a black box.

@joa-quim
Copy link
Member

That said, is this primarily because of gdal libs (i.e. code) being used in gmt,

Yes, and also because PROJ is mandatory in GDAL and we may want to use more things of PROJ directly (or indirectly).

@anbj
Copy link
Contributor

anbj commented Jan 24, 2022

That said, is this primarily because of gdal libs (i.e. code) being used in gmt,

Yes, and also because PROJ is mandatory in GDAL and we may want to use more things of PROJ directly (or indirectly).

Baking in PROJ with gmt (i.e. being able to use PROJ projections) sounds very attractive.

@maxrjones
Copy link
Member

I agree.

I think making GDAL a required dependency should be included in the changelog for anyone who's been building GMT without gdal.

We'll need to move libgdal-dev from the optional packages to the required packages in ci/install-dependencies-linux.sh, update the job name "Linux (without GDAL et al.)" in .github/workflows/build.yml, add gdal to the install steps in ci/vercel-docs.sh, and update all the GitHub wiki pages for gdal to be in the required rather than optional section. There's also a few places throughout the gmt docs where statements such as 'provided GMT was compiled
with GDAL support' need updating.

@PaulWessel
Copy link
Member Author

I have searched for GDAL in all GMT files and gone through manually and removed any mention of GDAL being optional or requiring special options. It should be all good to go as far as GMT is concerned. Once approved and merged we need to tend to the other issues that @meghanrjones outlined.

@PaulWessel PaulWessel changed the title WIP It is time to make GDAL a required prerequisite It is time to make GDAL a required prerequisite Jan 25, 2022
@PaulWessel PaulWessel merged commit 7faaa21 into master Jan 26, 2022
@PaulWessel PaulWessel deleted the gdal-required branch January 26, 2022 00:42
@PaulWessel
Copy link
Member Author

When should we change the information on the wiki? The GDAL being required is not true for the official releases yet.

@PaulWessel
Copy link
Member Author

Perhaps @meghanrjones can fix the CI script that does not use GDAL?

@maxrjones
Copy link
Member

When should we change the information on the wiki? The GDAL being required is not true for the official releases yet.

I think at release-time makes sense.

@maxrjones maxrjones mentioned this pull request Jan 26, 2022
4 tasks
weiji14 added a commit to GenericMappingTools/pygmt that referenced this pull request Jul 8, 2023
weiji14 added a commit to GenericMappingTools/pygmt that referenced this pull request Aug 8, 2023
Saving 3-band xarray.DataArray images to a temporary GeoTIFF
so that they can be plotted with grdimage.

* Refactor to use tempfile_from_image

Putting the temporary GeoTIFF creation logic in a dedicated
tempfile_from_image helper function, so that it can be reused
by other GMT modules besides grdimage. Also ensure that an
ImportError is raised when the `.rio` attribute cannot be found
when rioxarray is not installed.

* Let tilemap use tempfile_from_image func in virtualfile_from_data

Refactor Figure.tilemap to use the same tempfile_from_image
function that generates a temporary GeoTIFF file from the 3-band
xarray.DataArray images.

* Update docstring of grdimage with upstream GMT 6.4.0

Various updates from upstream GMT at
GenericMappingTools/gmt#6258,
GenericMappingTools/gmt@9069967,
GenericMappingTools/gmt#7260.

* Raise RuntimeWarning when input image dtype is not uint8

Plotting a non-uint8 dtype xarray.DataArray works in grdimage,
but the results will likely be incorrect. Warning the user about
the incorrect dtype, and suggest recasting to uint8 with 0-255 range,
e.g. using a histogram equalization function like
skimage.exposure.equalize_hist.

---------

Co-authored-by: Dongdong Tian <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Boring but important stuff for the core devs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants