Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix Python 3.7 compatibility #135

Merged
merged 2 commits into from
Jul 30, 2018
Merged

Fix Python 3.7 compatibility #135

merged 2 commits into from
Jul 30, 2018

Conversation

avalentino
Copy link
Contributor

Improve compatibility with Python 3.7

@avalentino
Regenerate cython extensions
3e76bfb 
Cython extensions (_fornav.cpp and _ll2cr.c) with Cython v0.28.2 to ensure
compatibility with Python 3.7.
@avalentino
Fix StopIteration handling
aa3c13b 
Compatibility with Python 3.7 and PEP 479.
@avalentino avalentino mentioned this pull request Jul 14, 2018
Closed
1 task
@coveralls
Copy link

coveralls commented Jul 14, 2018 •
edited
Loading

Coverage Status

Coverage remained the same at 85.561% when pulling aa3c13b on avalentino:python37-compat into 0a1e835 on pytroll:master.

@codecov
Copy link

codecov bot commented Jul 14, 2018 •
edited
Loading

Codecov Report

Merging #135 into master will not change coverage.
The diff coverage is 0%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #135   +/-   ##
=======================================
  Coverage   85.59%   85.59%           
=======================================
  Files          35       35           
  Lines        5984     5984           
=======================================
  Hits         5122     5122           
  Misses        862      862
Impacted Files Coverage Δ
pyresample/_multi_proc.py 42.3% <0%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0a1e835...aa3c13b. Read the comment docs.

@djhoese
Copy link
Member

djhoese commented Jul 14, 2018

Awesome thanks @avalentino. Just for historical purposes, what is the error you get with the StopIteration usage?

@avalentino
Copy link
Contributor Author

@djhoese error messages are:

======================================================================
ERROR: test_grid_filter2D (pyresample.test.test_geometry.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/projects/pyresample/pyresample/test/test_geometry.py", line 477, in test_grid_filter2D
    swath_def_f, data_f = grid_filter.filter(swath_def, data)
  File "/home/antonio/projects/pyresample/pyresample/geo_filter.py", line 79, in filter
    valid_index = self.get_valid_index(geometry_def)
  File "/home/antonio/projects/pyresample/pyresample/geo_filter.py", line 52, in get_valid_index
    x_coord, y_coord = proj(lons, lats, nprocs=self.nprocs)
  File "/home/antonio/projects/pyresample/pyresample/_spatial_mp.py", line 158, in __call__
    _run_jobs(_parallel_proj, proj_call_args, nprocs)
  File "/home/antonio/projects/pyresample/pyresample/_spatial_mp.py", line 202, in _run_jobs
    (ierr.value, err_msg.value.decode()))
RuntimeError: 2 errors in worker processes. Last one reported:
generator raised StopIteration

======================================================================
ERROR: test_latlons_mp (pyresample.test.test_grid.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/projects/pyresample/pyresample/test/test_grid.py", line 138, in test_latlons_mp
    res = self.area_def2.get_lonlats(nprocs=2)
  File "/home/antonio/projects/pyresample/pyresample/geometry.py", line 1308, in get_lonlats
    nprocs=nprocs)
  File "/home/antonio/projects/pyresample/pyresample/_spatial_mp.py", line 158, in __call__
    _run_jobs(_parallel_proj, proj_call_args, nprocs)
  File "/home/antonio/projects/pyresample/pyresample/_spatial_mp.py", line 202, in _run_jobs
    (ierr.value, err_msg.value.decode()))
RuntimeError: 2 errors in worker processes. Last one reported:
generator raised StopIteration

======================================================================
ERROR: test_resampled_image_mp (pyresample.test.test_grid.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/projects/pyresample/pyresample/test/test_grid.py", line 190, in test_resampled_image_mp
    target_def, source_def, data, nprocs=2, segments=1)
  File "/home/antonio/projects/pyresample/pyresample/grid.py", line 248, in get_resampled_image
    lons, lats = target_area_def.get_lonlats(nprocs)
  File "/home/antonio/projects/pyresample/pyresample/geometry.py", line 1308, in get_lonlats
    nprocs=nprocs)
  File "/home/antonio/projects/pyresample/pyresample/_spatial_mp.py", line 158, in __call__
    _run_jobs(_parallel_proj, proj_call_args, nprocs)
  File "/home/antonio/projects/pyresample/pyresample/_spatial_mp.py", line 202, in _run_jobs
    (ierr.value, err_msg.value.decode()))
RuntimeError: 2 errors in worker processes. Last one reported:
generator raised StopIteration

======================================================================
ERROR: test_gauss_multi_mp (pyresample.test.test_kd_tree.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/projects/pyresample/pyresample/test/test_kd_tree.py", line 361, in test_gauss_multi_mp
    nprocs=2, segments=1)
  File "/home/antonio/projects/pyresample/pyresample/kd_tree.py", line 179, in resample_gauss
    reduce_data=reduce_data, nprocs=nprocs, segments=segments, with_uncert=with_uncert)
  File "/home/antonio/projects/pyresample/pyresample/kd_tree.py", line 260, in _resample
    segments=segments)
  File "/home/antonio/projects/pyresample/pyresample/kd_tree.py", line 375, in get_neighbour_info
    nprocs=nprocs)
  File "/home/antonio/projects/pyresample/pyresample/kd_tree.py", line 529, in _query_resample_kdtree
    data_slice=data_slice, dtype=source_geo_def.dtype)
  File "/home/antonio/projects/pyresample/pyresample/geometry.py", line 1308, in get_lonlats
    nprocs=nprocs)
  File "/home/antonio/projects/pyresample/pyresample/_spatial_mp.py", line 158, in __call__
    _run_jobs(_parallel_proj, proj_call_args, nprocs)
  File "/home/antonio/projects/pyresample/pyresample/_spatial_mp.py", line 202, in _run_jobs
    (ierr.value, err_msg.value.decode()))
RuntimeError: 2 errors in worker processes. Last one reported:
generator raised StopIteration

======================================================================
ERROR: test_gauss_multi_mp_segments (pyresample.test.test_kd_tree.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/projects/pyresample/pyresample/test/test_kd_tree.py", line 381, in test_gauss_multi_mp_segments
    nprocs=2, segments=1)
  File "/home/antonio/projects/pyresample/pyresample/kd_tree.py", line 179, in resample_gauss
    reduce_data=reduce_data, nprocs=nprocs, segments=segments, with_uncert=with_uncert)
  File "/home/antonio/projects/pyresample/pyresample/kd_tree.py", line 260, in _resample
    segments=segments)
  File "/home/antonio/projects/pyresample/pyresample/kd_tree.py", line 375, in get_neighbour_info
    nprocs=nprocs)
  File "/home/antonio/projects/pyresample/pyresample/kd_tree.py", line 529, in _query_resample_kdtree
    data_slice=data_slice, dtype=source_geo_def.dtype)
  File "/home/antonio/projects/pyresample/pyresample/geometry.py", line 1308, in get_lonlats
    nprocs=nprocs)
  File "/home/antonio/projects/pyresample/pyresample/_spatial_mp.py", line 158, in __call__
    _run_jobs(_parallel_proj, proj_call_args, nprocs)
  File "/home/antonio/projects/pyresample/pyresample/_spatial_mp.py", line 202, in _run_jobs
    (ierr.value, err_msg.value.decode()))
RuntimeError: 2 errors in worker processes. Last one reported:
generator raised StopIteration

======================================================================
ERROR: test_nearest_mp (pyresample.test.test_kd_tree.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/projects/pyresample/pyresample/test/test_kd_tree.py", line 221, in test_nearest_mp
    self.area_def, 50000, nprocs=2, segments=1)
  File "/home/antonio/projects/pyresample/pyresample/kd_tree.py", line 99, in resample_nearest
    reduce_data=reduce_data, nprocs=nprocs, segments=segments)
  File "/home/antonio/projects/pyresample/pyresample/kd_tree.py", line 260, in _resample
    segments=segments)
  File "/home/antonio/projects/pyresample/pyresample/kd_tree.py", line 375, in get_neighbour_info
    nprocs=nprocs)
  File "/home/antonio/projects/pyresample/pyresample/kd_tree.py", line 529, in _query_resample_kdtree
    data_slice=data_slice, dtype=source_geo_def.dtype)
  File "/home/antonio/projects/pyresample/pyresample/geometry.py", line 1308, in get_lonlats
    nprocs=nprocs)
  File "/home/antonio/projects/pyresample/pyresample/_spatial_mp.py", line 158, in __call__
    _run_jobs(_parallel_proj, proj_call_args, nprocs)
  File "/home/antonio/projects/pyresample/pyresample/_spatial_mp.py", line 202, in _run_jobs
    (ierr.value, err_msg.value.decode()))
RuntimeError: 2 errors in worker processes. Last one reported:
generator raised StopIteration

----------------------------------------------------------------------
Ran 189 tests in 88.873s

FAILED (errors=6, skipped=1)

@djhoese djhoese merged commit b25045b into pytroll:master Jul 30, 2018
@ocefpaf ocefpaf mentioned this pull request Oct 1, 2018
Closed
@djhoese djhoese changed the title Python37 compat Fix Python 3.7 compatibility Oct 1, 2018
@djhoese djhoese added the bug label Oct 1, 2018
@avalentino avalentino deleted the python37-compat branch March 19, 2020 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@avalentino @coveralls @djhoese