Skip to content

Dataset to Dataset Transformations (Transmuting since 2019)

License

Notifications You must be signed in to change notification settings

mmochan/datacube-alchemist

 
 

Repository files navigation

datacube-alchemist - ODC Dataset to Dataset Converter

Copyright: © 2019, Geoscience Australia.

Version: 0.0.1

Test and build status

PURPOSE

Datacube Alchemst is a command line application for performing Dataset -> Dataset transformations in the context of an Open Data Cube system.

It uses a configuration file which specifies an input Product, a transformation to perform, and output parameters and destination.

Features

  • Writes output a Cloud Optimised GeoTIFFs
  • Easily run within a Docker Container
  • Parallelism using AWS SQS queues
  • Write output data to S3
  • Generates eo3 format dataset metadata, along with processing information
  • Configurable thumbnail generation
  • Pass any command line options as Environment Variables

INSTALLATION

You can build the docker image locally with Docker or Docker Compose. The commands are docker build --tag opendatacube/datacube-alchemist . or docker-compose build.

There's a Python setup file, so you can do pip3 install . in the root folder. You will need to ensure that the Open Data Cube and all its dependencies happily install though.

USAGE


To run some example processes you can use the Docker Compose file to create a local workspace. To start the workspace and run an example, you can do the following:

  • make up or docker-compose up to start the postgres and datacube-alchemist Docker containers
  • make initdb to initialise the ODC database (or see the Makefile for the specific command)
  • make metadata will add the eo-plus metadata that the Sentinel-2 example product needs
  • make product will add the Sentinel-2 product definition
  • make add-one-scene will index a single Sentinel-2 scene for us to process
  • make run-one-fc or make run-one-wofs will process a single Fractional Cover or Water Observations from Space scene and output the results to /tmp/alchemist locally

NOTES

LICENSE

Apache License 2.0

About

Dataset to Dataset Transformations (Transmuting since 2019)

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 89.7%
  • Dockerfile 5.3%
  • Makefile 5.0%