This repository contains Dockerfile of airflow for Docker's automated build published to the public Docker Hub Registry.
- Based on Debian Jessie official Image debian:jessie and uses the official Postgres as backend and RabbitMQ as queue
- Install Docker
- Install Docker Compose
- Following the Airflow release from Python Package Index
Pull the image from the Docker repository.
docker pull puckel/docker-airflow
For example, if you need to install Extra Packages, edit the Dockerfile and than build-it.
docker build --rm -t puckel/docker-airflow .
Start the stack (postgres, rabbitmq, airflow-webserver, airflow-scheduler airflow-flower & airflow-worker):
docker-compose up -d
If you want to use Ad hoc query, make sure you've configured connections: Go to Admin -> Connections and Edit "mysql_default" set this values (equivalent to values in airflow.cfg/docker-compose.yml) :
- Host : mysql
- Schema : airflow
- Login : airflow
- Password : airflow
Check Airflow Documentation
- Airflow: localhost:8080
- Flower: localhost:5555
- RabbitMQ: localhost:15672
When using OSX with boot2docker, use: open https://$(boot2docker ip):8080
docker exec dockerairflow_webserver_1 airflow backfill tutorial -s 2015-05-01 -e 2015-06-01
Easy scaling using docker-compose:
docker-compose scale worker=5
This can be used to scale to a multi node setup using docker swarm.
Fork, improve and PR. ;-)