Skip to content

mwaaas/docker-airflow-1

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

74 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

docker-airflow

CircleCI Docker Hub Docker Pulls Docker Stars

This repository contains Dockerfile of airflow for Docker's automated build published to the public Docker Hub Registry.

Informations

Installation

Pull the image from the Docker repository.

    docker pull puckel/docker-airflow

Build

For example, if you need to install Extra Packages, edit the Dockerfile and than build-it.

    docker build --rm -t puckel/docker-airflow .

Usage

Start the stack (postgres, rabbitmq, airflow-webserver, airflow-scheduler airflow-flower & airflow-worker):

    docker-compose up -d

If you want to use Ad hoc query, make sure you've configured connections: Go to Admin -> Connections and Edit "mysql_default" set this values (equivalent to values in airflow.cfg/docker-compose.yml) :

  • Host : mysql
  • Schema : airflow
  • Login : airflow
  • Password : airflow

Check Airflow Documentation

UI Links

When using OSX with boot2docker, use: open https://$(boot2docker ip):8080

Run the test "tutorial"

    docker exec dockerairflow_webserver_1 airflow backfill tutorial -s 2015-05-01 -e 2015-06-01

Scale the number of workers

Easy scaling using docker-compose:

    docker-compose scale worker=5

This can be used to scale to a multi node setup using docker swarm.

Wanna help?

Fork, improve and PR. ;-)

About

Simple Airbnb Airflow container

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Shell 100.0%