linksociety/flowg repository overview

⁠FlowG

FlowG is a log management platform that lets you ingest, transform, and query logs using a visual pipeline builder. It handles structured logs without requiring predefined schemas and relies on BadgerDB⁠ as its storage backend.

For more informations, consult the documentation⁠.

You wish to contribute? Please read our guidelines⁠ first.

⁠✨ Key Features

• Pipelines: Build and manage log processing pipelines visually, with support for transormations with VRL⁠ scripts and conditional routing to dedicated streams
• Dynamic Logs: Manage logs with varying structures without needing predefined schemas
• Interoperability: Configuration and log ingestion can entirely be done via the REST API
• Observability: Query and visualize logs in real-time in the integrated Web interface

⁠❓ Why should I use this?

FlowG's primary goal is to make log refinement and routing as easy as possible. As such, it relies on React Flow⁠ to help you build such pipelines with as little code as possible. For the actual code part, we use the Vector Remap Language⁠, which gives a solid base for log refinement.

It aims to replace tools like Logstash⁠ by integrating the feature right in the solution.

It also leverages BadgerDB⁠ which is a battle tested Key/Value database, with the right feature set to easily support indexing dynamically structured logs, as well as log compression.

⁠🚀 Usage

$ docker run -p 5080:5080 -v ./data:/data linksociety/flowg:latest

Then you can access:

• the WebUI at http://localhost:5080⁠
• the API documentation at http://localhost:5080/api/docs⁠

⁠📝 License

This software is released under the terms of the MIT License⁠