shinyorke/airflow repository overview
airflow-docker
This repository contains Dockerfile of apache-airflow for Docker's automated build published to the public Docker Hub Registry.
Informations
- Based on Python (3.6-stretch) official Image python:3.6-stretch
- Install Docker
- Install Docker Compose
- Following the Airflow release from Python Package Index
Installation
Pull the image from the Docker repository.
docker pull shinyorke/airflow
Build
docker build --rm -t shinyorke/airflow .
Settings
environments for airflow.cfg
sample:env_example
# User & HOME
AIRFLOW_USER=airflow
AIRFLOW_HOME=/usr/local/airflow
# Database(MySQL)
AIRFLOW_DB_CONN=mysql://airflow:password@localhost:3306/airflow
# Database(sqlite)
# AIRFLOW_DB_CONN=sqlite:////usr/local/airflow/airflow.db
# Airflow webserver settings
AIRFLOW_WEB_PROTOCOL=http
AIRFLOW_WEB_HOST=localhost
AIRFLOW_WEB_PORT=8080
AIRFLOW_WEB_WORKERS=4
# Airflow worker settings
AIRFLOW_WORKER=redis
AIRFLOW_WORKER_HOST=redis
AIRFLOW_WORKER_PORT=6379
AIRFLOW_WORKER_DATABASE=0
# Airflow base settings
# SequentialExecutor, LocalExecutor, CeleryExecutor
AIRFLOW_EXECUTOR=SequentialExecutor
AIRFLOW_EXAMPLES=False
AIRFLOW_AUTH=False
SQL_ALCHEMY_POOL_SIZE=5
SQL_ALCHEMY_POOL_RECYCLE=3600
# Application PATH
PYTHONPATH=/usr/local/airflow
Usage
webserver
docker run -p 8080:8080 --env-file=./env_example shinyorke/airflow webserver init
scheduler
docker run --env-file=./env_example shinyorke/airflow scheduler init
worker
docker run --env-file=./env_example shinyorke/airflow worker init
Sample(docker-compose)
airflow service example
- webserver
- scheduler
- worker
- backend
-
MySQL(airflow db)
-
redis(workder queue)
docker-compose -f docker-compose-example-db.yml up -d docker-compose -f docker-compose-example.yml build docker-compose -f docker-compose-example.yml up -d
-
Wanna help?
Fork, improve and PR. Sorry m(_ _)m
Maintainer
Shinichi Nakagawa
Tag summary
Content type
Image
Digest
Size
606.6 MB
Last updated
over 8 years ago
docker pull shinyorke/airflow