Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add reduce method to rasterize #336

Merged
merged 50 commits into from
Apr 1, 2023
Merged

add reduce method to rasterize #336

merged 50 commits into from
Apr 1, 2023

Conversation

rafaqz
Copy link
Owner

@rafaqz rafaqz commented Nov 17, 2022

adds reduce funciton argument to rasterize, fixes #333

@harithmorgadinho see if this works for you, ] add Rasters#reduce_rasterize should check out this branch

@harithmorgadinho
Copy link

@rafaqz thank you for doing this. I just tried running the rasterization with and without touching. The run without touching seems to be fine but with touching, there seems to be an issue with the pacific islands and the extent of the map. Check the attached figure.
Screenshot 2022-11-17 at 13 38 47

@rafaqz
Copy link
Owner Author

rafaqz commented Dec 4, 2022

So turns out its just one of the shapes, number 7081. And its not crossing 180, its some internal issue with that polygon and rasterize - like the rings are not all closed or rasterize isn't handling them all properly.

using Rasters, Shapefile, Plots
shp = Shapefile.Handle("SCALED_REPTILES.shp")
raster = rasterize(shp.shapes[7081]; fill=1, res=0.2, boundary=:touches)
plot(raster)

badgeom

Edit: fixed! line burning wasn't handling multi polygons properly. Will add it to this PR soon and get this merged.

fixed

@rafaqz
Copy link
Owner Author

rafaqz commented Dec 5, 2022

fixed

@rafaqz
Copy link
Owner Author

rafaqz commented Feb 5, 2023

@harithmorgadinho this seems to be competitive with fasterize now ;)

 using Rasters, BenchmarkTools, Shapefile, Plots
shppath = "/path/to/MAMMALS_TERRESTRIAL_ONLY/MAMMALS_TERRESTRIAL_ONLY.shp"
shptable = Shapefile.Table(shppath)
mammal_count = rasterize(count, shptable; res=1/6, boundary=:center);
plot(mammal_count);
savefig("mammal_count.png");

Here (I think?) count does what "sum" does in the fasterize readme benchmark. In this PR sum is summing a real column of the table or some other vector you pass in as fill, count is just counting the presence/absence of the polygons without any fill.

mammal_count

I'm not sure what machine fasterize is tested on but my laptop this runs faster than their benchmark of the mammals dataset. I will have to get it working in R to really compare:

julia> @time mammal_count = rasterize(count, shptable; res=1/6, boundary=:center);
  0.641720 seconds (272.59 k allocations: 2.244 GiB)

julia> @time mammal_count_touches = rasterize(count, shptable; res=1/6, boundary=:touches);
  1.195055 seconds (285.05 k allocations: 2.246 GiB)

And with :touches its nearly as fast

@codecov-commenter
Copy link

codecov-commenter commented Mar 27, 2023

Codecov Report

Merging #336 (af33672) into main (c4a8c83) will increase coverage by 4.43%.
The diff coverage is 89.22%.

@@            Coverage Diff             @@
##             main     #336      +/-   ##
==========================================
+ Coverage   75.64%   80.08%   +4.43%     
==========================================
  Files          40       41       +1     
  Lines        3310     3802     +492     
==========================================
+ Hits         2504     3045     +541     
+ Misses        806      757      -49     
Impacted Files Coverage Δ
src/Rasters.jl 66.66% <ø> (ø)
src/methods/aggregate.jl 93.28% <ø> (ø)
src/sources/rasterdatasources.jl 26.66% <0.00%> (ø)
src/show.jl 91.66% <50.00%> (ø)
src/table_ops.jl 75.00% <66.66%> (+65.27%) ⬆️
src/sectorlock.jl 77.50% <77.50%> (ø)
src/polygon_ops.jl 87.33% <87.39%> (+7.48%) ⬆️
src/methods/rasterize.jl 87.13% <88.56%> (-4.68%) ⬇️
src/methods/crop_extend.jl 89.24% <88.88%> (-0.65%) ⬇️
src/sources/gdal.jl 84.96% <91.30%> (+0.66%) ⬆️
... and 7 more

... and 2 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@rafaqz rafaqz force-pushed the reduce_rasterize branch 2 times, most recently from c0e9a37 to c080632 Compare April 1, 2023 15:22
@rafaqz rafaqz merged commit 964acca into main Apr 1, 2023
@rafaqz rafaqz deleted the reduce_rasterize branch April 1, 2023 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

rasterize by a specified function and column
3 participants