marcelmittelstaedt/airflow repository overview

This docker image serves the purpose of experimental usage and for educational purposes. It should not be used in a productive manner in any way.

Source Files: https://github.com/marcelmittelstaedt/BigData/tree/master/docker/airflow⁠

Used within lecture "Big Data", see:

• https://github.com/marcelmittelstaedt/BigData⁠ or
• http://wwwlehre.dhbw-stuttgart.de/~mittelstaedt/⁠ for more details.

Image Contains:

• Ubuntu 18.04 LTS
• Airflow 1.10.5
• PostgreSQL 10.10 (as Metadata Store)
• Hadoop 3.1.2 Client
• Hive 3.1.2 Client (Beeline and Hive)
• Spark/PySpark 2.3.4 Client

Setup to work with another Docker container on same network (marcelmittelstaedt/spark_base:latest).

Usage:

• Airflow and PostgreSQL start automatically on start/restart
• Airflow UI: http://X.X.X.X:8080⁠