Public | Automated Build

Last pushed: 8 months ago
Short Description
Docker images for the airflow
Full Description

airflow-docker



This repository contains Dockerfile of apache-airflow for Docker's automated build published to the public Docker Hub Registry.

Informations

Installation

Pull the image from the Docker repository.

    docker pull shinyorke/airflow

Build

    docker build --rm -t shinyorke/airflow .

Settings

environments for airflow.cfg

sample:env_example

# User & HOME
AIRFLOW_USER=airflow
AIRFLOW_HOME=/usr/local/airflow

# Database(MySQL)
AIRFLOW_DB_CONN=mysql://airflow:password@localhost:3306/airflow

# Database(sqlite)
# AIRFLOW_DB_CONN=sqlite:////usr/local/airflow/airflow.db

# Airflow webserver settings
AIRFLOW_WEB_PROTOCOL=http
AIRFLOW_WEB_HOST=localhost
AIRFLOW_WEB_PORT=8080
AIRFLOW_WEB_WORKERS=4

# Airflow worker settings
AIRFLOW_WORKER=redis
AIRFLOW_WORKER_HOST=redis
AIRFLOW_WORKER_PORT=6379
AIRFLOW_WORKER_DATABASE=0

# Airflow base settings
# SequentialExecutor, LocalExecutor, CeleryExecutor
AIRFLOW_EXECUTOR=SequentialExecutor
AIRFLOW_EXAMPLES=False
AIRFLOW_AUTH=False
SQL_ALCHEMY_POOL_SIZE=5
SQL_ALCHEMY_POOL_RECYCLE=3600

# Application PATH
PYTHONPATH=/usr/local/airflow

Usage

webserver

    docker run -p 8080:8080 --env-file=./env_example shinyorke/airflow webserver init

scheduler

    docker run --env-file=./env_example shinyorke/airflow scheduler init

worker

    docker run --env-file=./env_example shinyorke/airflow worker init

Sample(docker-compose)

airflow service example

  • webserver
  • scheduler
  • worker
  • backend

    • MySQL(airflow db)
    • redis(workder queue)

      docker-compose -f docker-compose-example-db.yml up -d
      docker-compose -f docker-compose-example.yml build
      docker-compose -f docker-compose-example.yml up -d
      

Wanna help?

Fork, improve and PR. Sorry m(_ _)m

Maintainer

Shinichi Nakagawa

Docker Pull Command
Owner
shinyorke