marcosschroh/faust-docker-example repository overview

⁠What is Faust

Faust is a stream processing library, porting the ideas from Kafka Streams to Python.

It is used at Robinhood to build high performance distributed systems and real-time data pipelines that process billions of events every day.

Faust provides both stream processing and event processing, sharing similarity with tools such as Kafka Streams, Apache Spark/Storm/Samza/Flink,

It does not use a DSL, it’s just Python! This means you can use all your favorite Python libraries when stream processing: NumPy, PyTorch, Pandas, NLTK, Django, Flask, SQLAlchemy, ++

Faust requires Python 3.6 or later for the new async/await syntax, and variable type annotations.

⁠Content

This image contains a Faust application with 3 "packages". Two packages are from Faust tutorials Page Views⁠ and Leader Election⁠, the other one is an example of how to integrate the schema-registry with Faust. For the full example, you can take a look here⁠

⁠Github Project

https://github.com/marcosschroh/faust-docker-compose-example⁠

⁠How to use this image

Because is a streaming processing we need to set up zookeeper and kafka and optionally schema-registry. You can find the description of these images in confluentinc⁠

I will recommend using docker-compose at the beginning, for example, https://github.com/marcosschroh/faust-docker-compose-example/blob/master/docker-compose.yaml⁠. You can run with docker run marcosschroh/faust-docker-example

⁠Environment Variables