4pdopensource/pafka-dev repository overview

⁠Pafka: PMem-Accelerated Kafka

Pafka is an evolved version of Apache Kafka developed by 4Paradigm. Kafka is an open-source distributed event streaming/message queue system for handling real-time data feeds efficiently and reliably. However, its performance (e.g., throughput) is constrained by the disk bandwidth, which further deteriorates due to the file system overhead.

Pafka equips Kafka with Intel® Optane™ Persistent Memory (PMem) support, which relies on the native pmdk libraries rather than treat PMem as a normal disk device. With careful design and implementation, Pafka can achieve 7.5 GB/s write throughput and 10 GB/s read throughput in terms of single-server performance.

Source code can be found here: https://github.com/4paradigm/pafka⁠