Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to create a single RasterStack from multiple multi-layer TIFF files with different extents while preserving the original band names.? #585

Closed
Jigyasu4indp opened this issue Dec 23, 2023 · 12 comments

Comments

@Jigyasu4indp
Copy link

Jigyasu4indp commented Dec 23, 2023

> using Pkg
> using Plots

> using Rasters, Arrow, ArchGDAL, DataFrames, Statistics
> using Rasters.Extents
> using Glob

# Read all .tif files and store rasters
> tif_files = glob("*.tif", ".")
> tif_files = tif_files[2:37]
> rasters = [Raster(tif_file) for tif_file in tif_files]

How to make a RasterStack of rasters while preserving the band names of tif_files.

@Jigyasu4indp Jigyasu4indp changed the title How to set the band names of the source files to match the original band names. How to set the band names in the processed RasterStack to match the original band names from the source files Dec 23, 2023
@rafaqz
Copy link
Owner

rafaqz commented Dec 23, 2023

This is too much code for a MWE and not runnable, so its hard to see what you want to do, try to post something short and runnable next time.

But, I'll guess: RasterStack has a layersfrom keyword that will convert a Raster dimension to layers:

rast = Raster(rand(10, 10, 5), (X(), Y(), Band([:a, :b, :c, :d, :e])))
st = RasterStack(rast; layersfrom=Band)

Which gives you a stack where the band lookup values are the layer names:

julia> st = RasterStack(rast; layersfrom=Band)
RasterStack with dimensions: X, Y
and 5 layers:
  :a Float64 dims: X, Y (10×10)
  :b Float64 dims: X, Y (10×10)
  :c Float64 dims: X, Y (10×10)
  :d Float64 dims: X, Y (10×10)
  :e Float64 dims: X, Y (10×10)

If your bands are just numbers the stack layers will be call :Band_1 etc.

If you want to combine multiple multiband rasters into a single stack, you can just merge stacks together.

@Jigyasu4indp Jigyasu4indp changed the title How to set the band names in the processed RasterStack to match the original band names from the source files How to create a single RasterStack from multiple multi-layer TIFF files with different extents while preserving the original band names.? Dec 23, 2023
@Jigyasu4indp
Copy link
Author

I updated the code.

@rafaqz
Copy link
Owner

rafaqz commented Dec 23, 2023

Ok, well exactly what I said then :)

@Jigyasu4indp
Copy link
Author

Here, "rasters[1]...rasters[n]" represents multilayered band data with different extents. How can I stack these into a single raster dataset?

@rafaqz
Copy link
Owner

rafaqz commented Dec 23, 2023

A RasterStack has a fixed extent. You can crop or extend them to combine - just splat them all into the function.
If the pixels dont match you need resample.

@Jigyasu4indp
Copy link
Author

Jigyasu4indp commented Dec 24, 2023

The commands layersfrom=Band and merge do not work for setting the band name to match that of the parent data and for stacking them, respectively. code:=

 > resampled_rasters[1]
31732×20640×91 Raster{Float32,3} with dimensions: 
  X Projected{Float64} LinRange{Float64}(28.9456, 114.617, 31732) ForwardOrdered Regular Intervals{Start} crs: WellKnownText,
  Y Projected{Float64} LinRange{Float64}(-13.3192, 42.4047, 20640) ForwardOrdered Regular Intervals{Start} crs: WellKnownText,
  Band Categorical{String} String[2018-05-31, 2018-05-30, …, 2018-11-02, 2018-11-01] Unordered
extent: Extent(X = (28.945555, 114.619383), Y = (-13.319213, 42.407447999999995), Band = (nothing, nothing))missingval: -Inf32crs: GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AXIS["Latitude",NORTH],AXIS["Longitude",EAST],AUTHORITY["EPSG","4326"]]
parent:
>  resampled_rasters[2]
31732×20640×27 Raster{Float32,3} with dimensions: 
  X Projected{Float64} LinRange{Float64}(28.9456, 114.617, 31732) ForwardOrdered Regular Intervals{Start} crs: WellKnownText,
  Y Projected{Float64} LinRange{Float64}(-13.3189, 42.4047, 20640) ForwardOrdered Regular Intervals{Start} crs: WellKnownText,
  Band Categorical{String} String[2019-04-30, 2019-04-29, …, 2019-04-02, 2019-04-01] Unordered
extent: Extent(X = (28.945555, 114.61989641724806), Y = (-13.318879049035672, 42.407447999999995), Band = (nothing, nothing))missingval: -Inf32crs: GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AXIS["Latitude",NORTH],AXIS["Longitude",EAST],AUTHORITY["EPSG","4326"]]
parent:
> gm = RasterStack(resampled_rasters[1][:, :, 1] , resampled_rasters[2][:, :, 2]; layersfrom=Band)
RasterStack with dimensions: 
  X Projected{Float64} LinRange{Float64}(28.9456, 114.617, 31732) ForwardOrdered Regular Intervals{Start} crs: WellKnownText,
  Y Projected{Float64} LinRange{Float64}(-13.3192, 42.4047, 20640) ForwardOrdered Regular Intervals{Start} crs: WellKnownText
and 2 layers:
  :layer1 Float32 dims: X, Y (31732×20640)
  :layer2 Float32 dims: X, Y (31732×20640)
> gm = merge(resampled_rasters[1][:, :, 1] , resampled_rasters[2][:, :, 2]; layersfrom=Band)
MethodError: no method matching merge(::Raster{Float32, 2, Tuple{X{Projected{Float64, LinRange{Float64, Int64}, DimensionalData.Dimensions.LookupArrays.ForwardOrdered, DimensionalData.Dimensions.LookupArrays.Regular{Float64}, DimensionalData.Dimensions.LookupArrays.Intervals{DimensionalData.Dimensions.LookupArrays.Start}, DimensionalData.Dimensions.LookupArrays.Metadata{Rasters.GDALsource, Dict{String, Any}}, WellKnownText{GeoFormatTypes.CRS}, Nothing, X{Colon}}}, Y{Projected{Float64, LinRange{Float64, Int64}, DimensionalData.Dimensions.LookupArrays.ForwardOrdered, DimensionalData.Dimensions.LookupArrays.Regular{Float64}, DimensionalData.Dimensions.LookupArrays.Intervals{DimensionalData.Dimensions.LookupArrays.Start}, DimensionalData.Dimensions.LookupArrays.Metadata{Rasters.GDALsource, Dict{String, Any}}, WellKnownText{GeoFormatTypes.CRS}, Nothing, Y{Colon}}}}, Tuple{Band{DimensionalData.Dimensions.LookupArrays.Categorical{String, Vector{String}, DimensionalData.Dimensions.LookupArrays.Unordered, DimensionalData.Dimensions.LookupArrays.NoMetadata}}}, Matrix{Float32}, Symbol, DimensionalData.Dimensions.LookupArrays.Metadata{Rasters.GDALsource, Dict{String, Any}}, Float32}, ::Raster{Float32, 2, Tuple{X{Projected{Float64, LinRange{Float64, Int64}, DimensionalData.Dimensions.LookupArrays.ForwardOrdered, DimensionalData.Dimensions.LookupArrays.Regular{Float64}, DimensionalData.Dimensions.LookupArrays.Intervals{DimensionalData.Dimensions.LookupArrays.Start}, DimensionalData.Dimensions.LookupArrays.Metadata{Rasters.GDALsource, Dict{String, Any}}, WellKnownText{GeoFormatTypes.CRS}, Nothing, X{Colon}}}, Y{Projected{Float64, LinRange{Float64, Int64}, DimensionalData.Dimensions.LookupArrays.ForwardOrdered, DimensionalData.Dimensions.LookupArrays.Regular{Float64}, DimensionalData.Dimensions.LookupArrays.Intervals{DimensionalData.Dimensions.LookupArrays.Start}, DimensionalData.Dimensions.LookupArrays.Metadata{Rasters.GDALsource, Dict{String, Any}}, WellKnownText{GeoFormatTypes.CRS}, Nothing, Y{Colon}}}}, Tuple{Band{DimensionalData.Dimensions.LookupArrays.Categorical{String, Vector{String}, DimensionalData.Dimensions.LookupArrays.Unordered, DimensionalData.Dimensions.LookupArrays.NoMetadata}}}, Matrix{Float32}, Symbol, DimensionalData.Dimensions.LookupArrays.Metadata{Rasters.GDALsource, Dict{String, Any}}, Float32}; layersfrom::UnionAll)
> gm = RasterStack(resampled_rasters[1] , resampled_rasters[2], dims = Band)
DimensionMismatch: Found both lengths 91 and 27 for Band.

Stacktrace:
  [1] _dimsizeerror(a::Band{DimensionalData.Dimensions.LookupArrays.Categorical{String, Vector{String}, DimensionalData.Dimensions.LookupArrays.Unordered, DimensionalData.Dimensions.LookupArrays.NoMetadata}}, b::Band{DimensionalData.Dimensions.LookupArrays.Categorical{String, Vector{String}, DimensionalData.Dimensions.LookupArrays.Unordered, DimensionalData.Dimensions.LookupArrays.NoMetadata}})
    @ DimensionalData.Dimensions /storage/brno12-cerit/home/ujjwal4r/.julia/packages/DimensionalData/R7veM/src/Dimensions/primitives.jl:741
  [2] #_comparedims#75

@rafaqz
Copy link
Owner

rafaqz commented Dec 24, 2023

You need to make stacks from each one separately, then use merge on the stacks.

A RasterStack doesn't use layersfrom if you pass in multiple Raster. Maybe it should? I actually never thought of that being a use-case.

@Jigyasu4indp
Copy link
Author

Jigyasu4indp commented Dec 24, 2023

"merge" is also not working.

>a =  RasterStack(resampled_rasters[1])
RasterStack with dimensions: 
  X Projected{Float64} LinRange{Float64}(28.9456, 114.617, 31732) ForwardOrdered Regular Intervals{Start} crs: WellKnownText,
  Y Projected{Float64} LinRange{Float64}(-13.3192, 42.4047, 20640) ForwardOrdered Regular Intervals{Start} crs: WellKnownText,
  Band Categorical{String} String[2018-05-31, 2018-05-30, …, 2018-11-02, 2018-11-01] Unordered
and 1 layer:
  :layer1 Float32 dims: X, Y, Band (31732×20640×91)

with metadata Metadata{Rasters.GDALsource} of Dict{String, Any} with 4 entries:
  "units"    => ""
  "offset"   => 0.0
  "filepath" => "./sentinel_2018_daily_rast.tif"
  "scale"    => 1.0
>b =  RasterStack(resampled_rasters[b])
RasterStack with dimensions: 
  X Projected{Float64} LinRange{Float64}(28.9456, 114.617, 31732) ForwardOrdered Regular Intervals{Start} crs: WellKnownText,
  Y Projected{Float64} LinRange{Float64}(-13.3192, 42.4047, 20640) ForwardOrdered Regular Intervals{Start} crs: WellKnownText,
  Band Categorical{String} String[2018-05-31, 2018-05-30, …, 2018-11-02, 2018-11-01] Unordered
and 1 layer:
  :layer1 Float32 dims: X, Y, Band (31732×20640×91)

with metadata Metadata{Rasters.GDALsource} of Dict{String, Any} with 4 entries:
  "units"    => ""
  "offset"   => 0.0
  "filepath" => "./sentinel_2018_daily_rast.tif"
  "scale"    => 1.0
> titu = merge(a, b)
RasterStack with dimensions: 
  X Projected{Float64} LinRange{Float64}(28.9456, 114.617, 31732) ForwardOrdered Regular Intervals{Start} crs: WellKnownText,
  Y Projected{Float64} LinRange{Float64}(-13.3192, 42.4047, 20640) ForwardOrdered Regular Intervals{Start} crs: WellKnownText,
  Band Categorical{String} String[2018-05-31, 2018-05-30, …, 2018-11-02, 2018-11-01] Unordered
and 1 layer:
  :layer1 Float32 dims: X, Y, Band (31732×20640×91)

with metadata Metadata{Rasters.GDALsource} of Dict{String, Any} with 4 entries:
  "units"    => ""
  "offset"   => 0.0
  "filepath" => "./sentinel_2018_daily_rast.tif"
  "scale"    => 1.0

"titu" should have 118 layers, but it only has 91 layers.

@rafaqz
Copy link
Owner

rafaqz commented Dec 24, 2023 via email

@Jigyasu4indp
Copy link
Author

Thanks, it's working!

@Jigyasu4indp
Copy link
Author

  1. How to manually set the names for RasterStack data because using 'set(RasterStack, Band => band_names)' is not working.

  2. Could you please share well-documented Raster package commands along with details and examples, similar to the 'terra' package in R

@rafaqz
Copy link
Owner

rafaqz commented Dec 24, 2023

One day... This is just one of very many packages I maintain and try to share with others.

R spatial data is decades old. In Julia we are just putting things together now. There is no possibility of our docs matching the R docs in the next five years, let alone soon - unless a whole lot more people start helping.

(please dont make requests like 2 again - its not reasonable. This is a free package and I'm helping you for free - are you going to help me in some way in return?)

With set you are just guessing at something you hope will work. Thats not a good approach to programming - try to understand what it does.

You are using set to modify a Band dimension. But check dims(rasterstack). There is no Band dimension.

Why don't you change the names when they are the Band dimension of a Raster ? Its harder to change stack layer names.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants