This repo demonstrates how to build a Zarr-based data cube from Sentinel 2 L2A Data in the AWS Open Data Program.
Note: This repo is for demonstration purposes only. It does not aspire to be a maintained package. If you want to build on top of it, fork this repo and modify it to your needs.
License: Apache 2.0
Supported Serverless Backends
- Coiled Functions
- Modal
- LithOps (AWS Lambda Executor)
Supported Storage Locations
- Arraylake - Earthmover's data lake platform for Zarr data
- Any fsspec-compatible cloud storage location (e.g. S3)
% python src/main.py --help
Usage: main.py [OPTIONS]
Options:
--start-date [%Y-%m-%d|%Y-%m-%dT%H:%M:%S|%Y-%m-%d %H:%M:%S]
Start date for the data cube. Everything but
year and month will be ignored. [required]
--end-date [%Y-%m-%d|%Y-%m-%dT%H:%M:%S|%Y-%m-%d %H:%M:%S]
Start date for the data cube. Everything but
year and month will be ignored. [required]
--bbox <FLOAT FLOAT FLOAT FLOAT>...
Bounding box for the data cube in lat/lon.
(min_lon, min_lat, max_lon, max_lat)
[required]
--time-frequency-months INTEGER RANGE
Temporal sampling frequency in months.
[1<=x<=24]
--resolution FLOAT Spatial resolution in degrees. [default:
0.0002777777777777778]
--chunk-size INTEGER Zarr chunk size for the data cube.
[default: 1200]
--bands TEXT Bands to include in the data cube. Must
match band names from odc.stac.load
[default: red, green, blue]
--varname TEXT The name of the variable to use in the Zarr
data cube. [default: rgb_median]
--epsg [4326] EPSG for the data cube. Only 4326 is
supported at the moment. [default: 4326]
--serverless-backend [coiled|modal|lithops]
[required]
--storage-backend [arraylake|fsspec]
[default: arraylake; required]
--arraylake-repo-name TEXT
--arraylake-bucket-nickname TEXT
--fsspec-uri TEXT
--limit INTEGER
--debug
--initialize / --no-initialize Initialize the Zarr store before processing.
--help Show this message and exit.
Vermont Datacube (monthly for 4 years), store in Arraylake
python src/main.py --start-date 2020-01-01 --end-date 2023-12-31 --bbox -73.43 42.72 -71.47 45.02 \
--arraylake-repo-name "earthmover-demos/sentinel-datacube-Vermont" --arraylake-bucket-nickname earthmover-demo-bucket \
--storage-backend arraylake --serverless-backend lithops
All of South America, one single month, store in Arraylake
python src/main.py --start-date 2020-01-01 --end-date 2020-01-31 --bbox -82 -56 -34 13 --chunk-size 1800 \
--arraylake-repo-name "earthmover-demos/sentinel-datacube-South-America" --arraylake-bucket-nickname earthmover-demo-bucket
--storage-backend arraylake --serverless-backend lithops
All of South America, 12 months, store in plain S3 bucket
python src/main.py --start-date 2020-01-01 --end-date 2020-12-31 --bbox -82 -56 -34 13 \
--storage-backend fsspec --fsspec-uri s3:https://earthmover-sample-data/zarr-datacubes/South-America\
--serverless-backend modal
lithops runtime build -f PipDockerfile -b aws_lambda serverless-datacube