This image ships fewer packages additions for puckel/docker-airflow image to avoid on-build runtime for faster deployments.
Learn about Apache Airflow at https://airflow.apache.org/
Packages: apache-airflow[crypto], numpy, psycopg2-binary, boto3, docker, fastparquet>=0.2.1, pandas>=0.24.1, pyarrow>=0.12.1, pyspark, scipy, xlrd, pendulum, python-slugify
There is a sub-tag that installs few python packages for scrapy usage.
Packages: loginform, scrapy>=1.6, scrapy_proxies, scrapy-fake-useragent
| origin | tag | alias |
|---|---|---|
puckel/docker-airflow:latest | latest, scrapy | - |
puckel/docker-airflow:1.10.2 | 1.10.2, 1.10.2-scrapy | - |
# docker-compose.yml
version: '3.1'
volumes:
postgres:
services:
airflow:
container_name: airflow
image: drentrega/airflow:1.10.2
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./src/dags:/usr/local/airflow/dags:ro
- ./src/plugins:/usr/local/airflow/plugins:ro
restart: always
command: webserver
depends_on:
- postgres
links:
- postgres
- api
expose:
- 8080
environment:
- AIRFLOW__CORE__FERNET_KEY={create-your-secret-key}
- LOAD_EX=n
- EXECUTOR=Local
- POSTGRES_HOST=postgres
- POSTGRES_PORT=5432
- POSTGRES_USER=airflow
- POSTGRES_PASSWORD=secret
- POSTGRES_DB=airflow
postgres:
container_name: sovereign-postgres
image: postgres:9.6-alpine
volumes:
- postgres:/var/lib/postgresql/data
expose:
- 5432
ports:
- "5432:5432"
environment:
- POSTGRES_DB=airflow
- POSTGRES_USER=airflow
- POSTGRES_PASSWORD=secret
Content type
Image
Digest
Size
441.9 MB
Last updated
over 7 years ago
docker pull drentrega/airflow