Apache Airflow container with mysql backend and multiple workers.
1.4K
Airflow v1.9 with:
Change s3 bucket name at S3_LOG_FOLDER in log_config.py.
Run MySQL Container.
docker run --name airflow-db -e MYSQL_ROOT_PASSWORD=airflow-db -d -v /docker_volumes/airflow-db:/var/lib/mysql mysql:latest
loging mysql and make database named airflow.
docker exec -it airflow-db bash
mysql -u root -p
create database airflow
docker run --name airflow-broker -d redis:4-alpine
docker run --name airflow-scheduler --link airflow-db:mysql --link airflow-broker:redis -v /path/to/dags/airflow/dags:/airflow/dags -d drunkar/airflow-alpine
docker run --name airflow-webserver --link airflow-db:mysql --link airflow-broker:redis -v /path/to/dags/airflow/dags:/airflow/dags -p 8080:8080 -d --entrypoint="/root/entrypoint_webserver.sh" drunkar/airflow-alpine
docker run --name airflow-worker --link airflow-db:mysql --link airflow-broker:redis -v /path/to/dags/airflow/dags:/airflow/dags -d --entrypoint="/root/entrypoint_worker.sh" drunkar/airflow-alpine
docker run --name airflow-flower --link airflow-broker:redis -p 5555:5555 -d --entrypoint="/root/entrypoint_flower.sh" drunkar/airflow-alpine
Conn Id: my_conn_S3
Conn Type: S3
Extra: {"aws_access_key_id":"your_aws_key_id", "aws_secret_access_key": "your_aws_secret_key"}
Then you can access to http://localhost:8080.
Content type
Image
Digest
Size
455.5 MB
Last updated
over 8 years ago
docker pull drunkar/airflow-alpine