Public | Automated Build

Last pushed: 9 days ago
Short Description
Airflow is a system to programmatically author, schedule and monitor data pipelines.
Full Description

Airflow Dockerfile

Circle CI :

ImageLayers :

This repository contains Dockerfile of airflow for Docker's automated build published to the public Docker Hub Registry.

Informations

Installation

    docker pull camil/airflow
    cd ~
    git clone https://github.com/camilb/docker-airflow.git

Build

For example, if you need to install Extra Packages, edit the Dockerfile and than build-it.

    cd ~/docker-airflow
    docker build --rm -t camil/airflow .

Usage

Start the stack (mysql, rabbitmq, airflow-webserver, airflow-scheduler airflow-flower & airflow-worker) :

    cd ~/docker-airflow
    docker-compose up -d

If you want to use Ad hoc query, make sure you've configured connections :
Go to Admin -> Connections and Edit "mysql_default" set this values (equivalent to values in airflow.cfg/docker-compose.yml) :

  • Host : mysql
  • Schema : airflow
  • Login : airflow
  • Password : airflow

Check Airflow Documentation

UI Links

(with boot2docker, use: open http://$(boot2docker ip):8080)

Run the test "tutorial"

    docker exec airflow_webserver_1 airflow backfill tutorial -s 2016-01-01 -e 2016-02-01

Web Authentication: create user

    docker exec -it airflow_webserver_1 python

Python 2.7.10 (default, Nov 20 2015, 05:21:22) [GCC 4.9.2] on linux2 Type "help", "copyright", "credits" or "license" for more information.

>>>

import airflow
from airflow import models, settings
from airflow.contrib.auth.backends.password_auth import PasswordUser
user = PasswordUser(models.User())
user.username = 'admin'
user.email = 'admin@admin.net'
user.password = 'secret'
session = settings.Session()
session.add(user)
session.commit()
session.close()
exit()

Wanna help?

Fork, improve and PR. ;-)

Docker Pull Command
Owner
camil
Source Repository

Comments (0)