Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reinstall optional geopandas dependency on CI tests #1331

Merged
merged 4 commits into from
Jun 18, 2021

Conversation

weiji14
Copy link
Member

@weiji14 weiji14 commented Jun 14, 2021

Description of proposed changes

Add geopandas as a dependency again on the latest NEP29 Python 3.9/NumPy 1.20 tests. Also unpin gdal version on ci_tests.yaml so that tests run on gdal>=3.3, and include geopandas in the conda environment.yml file.

This reverts the workaround in #1290 (comment) since conda-forge/fiona-feedstock#179 has been merged (so geopandas/fiona with gdal 3.3 can be installed from conda-forge). Note: Merge this after conda-forge/gmt-feedstock#154

Reminders

  • Run make format and make check to make sure the code follows the style guide.
  • Add tests for new features or tests that would have caught the bug that you're fixing.
  • Add new public functions/methods/classes to doc/api/index.rst.
  • Write detailed docstrings for all functions/methods.
  • If adding new functionality, add an example to docstrings or tutorials.

Slash Commands

You can write slash commands (/command) in the first line of a comment to perform
specific operations. Supported slash commands are:

  • /format: automatically format and lint the code
  • /test-gmt-dev: run full tests on the latest GMT development version

Add `geopandas` as a dependency again on the latest NEP29
Python 3.9/NumPy 1.20 tests. Also unpin gdal version on
ci_tests.yaml so that tests run on gdal>=3.3.
@weiji14 weiji14 added the maintenance Boring but important stuff for the core devs label Jun 14, 2021
@weiji14 weiji14 added this to the 0.4.0 milestone Jun 14, 2021
@weiji14 weiji14 self-assigned this Jun 14, 2021
@@ -83,7 +83,7 @@ jobs:
# Install dependencies from conda-forge
- name: Install dependencies
run: |
conda install ninja cmake libblas libcblas liblapack fftw gdal=3.2 geopandas \
conda install ninja cmake libblas libcblas liblapack fftw gdal geopandas \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unpinning gdal here should fix the failing GMT Dev Tests on Windows (that have been failing for months now, e.g. at https://github.com/GenericMappingTools/pygmt/runs/2811637685?check_suite_focus=true#step:16:18) by installing the GMT 6.3.0 dev version. However, this will need to wait until conda-forge/gmt-feedstock#154 is merged.

@@ -52,7 +52,7 @@ jobs:
optional-packages: ''
- python-version: 3.9
numpy-version: '1.20'
optional-packages: '' # 'geopandas'
optional-packages: 'geopandas'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like geopandas on conda-forge split into two packages in conda-forge/geopandas-feedstock#90. There is geopandas-base (light version) and geopandas (full version). An option might be to install geopandas-base and fiona (needed for doing I/O, c.f. #1000) instead of the full geopandas package (which brings in stuff like rtree and mapclassify that we probably don't need). Or we could keep things as is and just install geopandas since that's what most users would do.

@weiji14
Copy link
Member Author

weiji14 commented Jun 14, 2021

There are still some package conflicts...

UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:

Specifications:

  - codecov -> python[version='2.7.*|3.5.*|3.6.*']
  - pytest-mpl -> python[version='2.7.*|3.4.*|3.5.*|3.6.*']

Your python: python=3.9

If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.

The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package six conflicts for:
pytest-cov -> pytest[version='>=4.6'] -> six[version='>=1.10.0']
numpy=1.20 -> mkl-service[version='>=2.3.0,<3.0a0'] -> six
sphinx-gallery -> sphinx[version='>=1.8.3'] -> six[version='>=1.5']
pytest-mpl -> pytest -> six[version='>=1.10.0']
ipython -> prompt-toolkit[version='>=2.0.0,!=3.0.0,!=3.0.1,<3.1.0'] -> six[version='>=1.9.0']
pandas -> python-dateutil[version='>=2.7.3'] -> six[version='>=1.5']
packaging -> six
pytest -> more-itertools[version='>=4.0.0'] -> six[version='>=1.0.0,<2.0.0']
geopandas -> six
dvc -> configobj[version='>=5.0.6'] -> six[version='>=1.12,<2.0|>=1.13.0|>=1.5|>=1.13.0,<2dev|>=1.6.1,<2dev|>=1.6.1,<2']
geopandas -> fiona -> six[version='>=1.7']
pytest -> six[version='>=1.10.0']


Package pandas conflicts for:
geopandas -> mapclassify[version='>=2.2.0'] -> pandas[version='>=1.0']
geopandas -> pandas[version='>=0.23|>=0.24']
dvc -> pyarrow[version='>=0.17.1'] -> pandas
xarray -> dask[version='>=0.9'] -> pandas[version='>=0.19.0|>=0.21.0|>=0.23.0|>=0.25.0']
xarray -> pandas[version='>=0.18.0|>=0.18|>=0.19.2|>=0.24|>=0.25|>=1.0']

Package typing conflicts for:
dvc -> ruamel.yaml[version='>=0.16.1'] -> typing[version='>=3.7.4']
sphinx-gallery -> sphinx[version='>=1.8.3'] -> typing

Package libedit conflicts for:
python=3.9 -> sqlite[version='>=3.33.0,<4.0a0'] -> libedit[version='>=3.1.20191231,<3.2.0a0']
netcdf4 -> openssh -> libedit[version='>=3.1.20191231,<3.2.0a0']

Package freetype conflicts for:
geopandas -> matplotlib-base -> freetype[version='2.6.*|>=2.10.2,<3.0a0|>=2.10.4,<3.0a0|>=2.9.1,<3.0a0|>=2.8,<2.9.0a0']
pytest-mpl -> matplotlib-base -> freetype[version='2.5.*|2.6.*|2.7|2.7.*|2.7|2.8.*|2.8.1|2.8.1.*|>=2.10.2,<3.0a0|>=2.10.4,<3.0a0|>=2.9.1,<3.0a0|>=2.8.1,<2.9.0a0|>=2.10.3,<3.0a0|>=2.8,<2.9.0a0']
sphinx-gallery -> matplotlib-base -> freetype[version='2.5.*|2.6.*|2.7|2.7.*|2.7|2.8.*|2.8.1|2.8.1.*|>=2.10.2,<3.0a0|>=2.10.4,<3.0a0|>=2.9.1,<3.0a0|>=2.8.1,<2.9.0a0|>=2.10.3,<3.0a0|>=2.8,<2.9.0a0']
gmt=6.2.0 -> ffmpeg -> freetype[version='2.8.1|2.8.1.*|>=2.10.4,<3.0a0|>=2.9.1,<3.0a0|>=2.8.1,<2.9.0a0|>=2.8.1,<2.8.2.0a0|>=2.8,<2.9.0a0']

Package blas conflicts for:
numpy=1.20 -> mkl_fft -> blas=[build=mkl]
numpy=1.20 -> blas[version='*|1.0',build='mkl|openblas']

Package urllib3 conflicts for:
dvc -> dulwich[version='>=0.20.23'] -> urllib3[version='>=1.21.1,<1.25|>=1.21.1,<1.26,!=1.25.0,!=1.25.1|>=1.21.1,<1.27']
codecov -> requests[version='>=2.7.9'] -> urllib3[version='>=1.21.1,<1.22|>=1.21.1,<1.23|>=1.21.1,<1.24|>=1.21.1,<1.25|>=1.21.1,<1.26,!=1.25.0,!=1.25.1|>=1.21.1,<1.27']
geopandas -> pysal -> urllib3[version='<1.25|<1.26']

Package numpy-base conflicts for:
numpy=1.20 -> numpy-base[version='1.20.1|1.20.1|1.20.1|1.20.1|1.20.1|1.20.1|1.20.2',build='py39h34387ca_0|py38h7d8b39e_0|py39h7d8b39e_0|py38he2ba247_0|py37he2ba247_0|py39hfae3a4d_0|py37hfae3a4d_0|py39he2ba247_0|py38hfae3a4d_0|py38h34387ca_0|py37h7d8b39e_0|py37h34387ca_0']
numpy=1.20 -> mkl_fft -> numpy-base[version='>=1.0.14,<2.0a0|>=1.0.6,<2.0a0|>=1.0.2,<2.0a0|>=1.0.4,<2.0a0']

Package configparser conflicts for:
dvc -> configparser[version='>=3.5.0']
dvc -> importlib-metadata -> configparser[version='>=3.5']

Package mpich conflicts for:
netcdf4 -> hdf5=[build=mpi_mpich_*] -> mpich[version='3.2.*|>=3.2,<3.3.0a0|>=3.3,<3.4.0a0']
netcdf4 -> mpich[version='>=3.3,<4.0.0a0|>=3.4,<4.0.0a0']

Package blis conflicts for:
gmt=6.2.0 -> libblas[version='>=3.8.0,<4.0a0'] -> blis[version='0.5.1.*|>=0.5.2,<0.5.3.0a0|>=0.6.0,<0.6.1.0a0|>=0.6.1,<0.6.2.0a0|>=0.7.0,<0.7.1.0a0|>=0.8.0,<0.8.1.0a0|>=0.8.1,<0.8.2.0a0']
numpy=1.20 -> libblas[version='>=3.8.0,<4.0a0'] -> blis[version='0.5.1.*|>=0.5.2,<0.5.3.0a0|>=0.6.0,<0.6.1.0a0|>=0.6.1,<0.6.2.0a0|>=0.7.0,<0.7.1.0a0|>=0.8.0,<0.8.1.0a0|>=0.8.1,<0.8.2.0a0']

Package hdf5 conflicts for:
netcdf4 -> libnetcdf=4.4 -> hdf5=1.8.17
netcdf4 -> hdf5[version='*|*|*|>=1.10.6,<1.10.7.0a0|>=1.10.6,<1.10.7.0a0|>=1.10.6,<1.10.7.0a0|>=1.10.5,<1.10.6.0a0|>=1.10.5,<1.10.6.0a0|>=1.10.5,<1.10.6.0a0|>=1.10.4,<1.10.5.0a0|>=1.10.3,<1.10.4.0a0|1.10.2.*|>=1.10.2,<1.10.3.0a0|1.10.1.*|1.10.1|1.8.18|1.8.18.*|1.8.17|1.8.17.*|1.8.15.*|>=1.8.20,<1.9.0a0|>=1.8.18,<1.8.19.0a0|>=1.10.1,<1.10.2.0a0',build='mpi_mpich_*|mpi_openmpi_*|mpi_openmpi_*|nompi_*|mpi_openmpi_*|mpi_mpich_*']

Package enum34 conflicts for:
ipython -> traitlets[version='>=4.2'] -> enum34
geopandas -> fiona -> enum34

Error: Process completed with exit code 1.

@maxrjones maxrjones mentioned this pull request Jun 15, 2021
27 tasks
@weiji14 weiji14 marked this pull request as ready for review June 16, 2021 01:04
@weiji14
Copy link
Member Author

weiji14 commented Jun 16, 2021

Looks like the package conflicts magically resolved itself 🙌 PyGMT and geopandas installation using conda should now work!

@seisman seisman added the final review call This PR requires final review and approval from a second reviewer label Jun 16, 2021
@maxrjones
Copy link
Member

Why is geopandas included for the CI tests but not for the development dependencies in environment.yml?

@weiji14
Copy link
Member Author

weiji14 commented Jun 17, 2021

Why is geopandas included for the CI tests but not for the development dependencies in environment.yml?

Good question. Geopandas is an optional dependency, and isn't required for development. But I could add it to the environment.yml file if developers prefer to have it in their conda env? Or they could install it using conda install geopandas afterwards themselves.

@maxrjones
Copy link
Member

Why is geopandas included for the CI tests but not for the development dependencies in environment.yml?

Good question. Geopandas is an optional dependency, and isn't required for development. But I could add it to the environment.yml file if developers prefer to have it in their conda env? Or they could install it using conda install geopandas afterwards themselves.

I think if geopandas gallery examples will be suggested as good first issues for the upcoming sprint, it would be better to include it so that setup is as simple as possible for new contributors.

@weiji14
Copy link
Member Author

weiji14 commented Jun 17, 2021

I think if geopandas gallery examples will be suggested as good first issues for the upcoming sprint, it would be better to include it so that setup is as simple as possible for new contributors.

Makes sense, I'll add it in then. The Vercel continuous documentation actually installs from the environment.yml too (see package.json) so we'll need it for the gallery examples in the future. Edit: done at 35562cd

@weiji14 weiji14 merged commit a0b26b4 into master Jun 18, 2021
@weiji14 weiji14 deleted the reinstall-geopandas branch June 18, 2021 01:18
@maxrjones maxrjones removed the final review call This PR requires final review and approval from a second reviewer label Aug 16, 2021
sixy6e pushed a commit to sixy6e/pygmt that referenced this pull request Dec 21, 2022
…ols#1331)

Add `geopandas` as a dependency again on the latest NEP29
Python 3.9/NumPy 1.20 tests. Also unpin gdal version on
ci_tests.yaml so that tests run on gdal>=3.3, and include
geopandas in the conda environment.yml file.

Co-authored-by: Meghan Jones <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Boring but important stuff for the core devs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants