Public | Automated Build

Last pushed: a year ago
Short Description
Airflow DAG engine with postgresql and rabbitmq.
Full Description

Airflow Dockerfile

This repository contains Dockerfile of airflow for Docker's automated build published to the public Docker Hub Registry.

Information

Installation

    docker pull voyagerdata/airflow
    cd ~
    git clone https://github.com/voyager-data/docker-airflow.git

Build

For example, if you need to install Extra Packages, edit the Dockerfile and than build-it.

    cd ~/docker-airflow
    docker build --rm -t voyagerdata/airflow .

Usage

Start the stack (postgres, rabbitmq, airflow-webserver, airflow-scheduler airflow-flower & airflow-worker) :

    cd ~/docker-airflow
    docker-compose up -d

If you want to use Ad hoc query, make sure you've configured connections :
Go to Admin -> Connections and Edit "mysql_default" set this values (equivalent to values in airflow.cfg/docker-compose.yml) :

  • Host : postgres
  • Schema : postgres
  • Login : airflow
  • Password : airflow

Check Airflow Documentation

UI Links

(with boot2docker, use: open http://$(boot2docker ip):8080)

Run the test "tutorial"

    docker exec airflow_webserver_1 airflow backfill tutorial -s 2016-01-01 -e 2016-02-01

Web Authentication: create user

    docker exec -it airflow_webserver_1 python

Python 2.7.10 (default, Nov 20 2015, 05:21:22) [GCC 4.9.2] on linux2 Type "help", "copyright", "credits" or "license" for more information.

>>>

import airflow
from airflow import models, settings
from airflow.contrib.auth.backends.password_auth import PasswordUser
user = PasswordUser(models.User())
user.username = 'admin'
user.email = 'admin@admin.net'
user.password = 'secret'
session = settings.Session()
session.add(user)
session.commit()
session.close()
exit()

Wanna help?

Fork, improve and PR. ;-)

Docker Pull Command
Owner
voyagerdata
Source Repository

Comments (0)