Public | Automated Build

Last pushed: a year ago
Short Description
Tool that listen on a given port and throw all data received to a kafka topic
Full Description

n2kafka

Network to kafka translator. It (currently) support conversion from tcp/udp raw
sockets and HTTP POST to kafka messages, doing message-processing if you need
to.

Setup

To use it, you only need to do a typical ./configure && make && make install

Usage

Basic usage

In order to send raw tcp messages from port 2056, to mymessages topic, using
40 threads, you need to use this config file:

{
    "listeners":[
        {"proto":"tcp","port":2056,"num_threads":40}
    ],
    "brokers":"localhost",
    "topic":"mymessages"
}

And launch n2kafka using ./n2kafka <config_file>. You can also use udp and
http as proto values. If you want to listen in different ports, you can add as
many listeners as you want.

Recommended config parameters

You can also use this parameters in config json root to improve n2kafka
behavior:

  • "blacklist":["192.168.101.3"], that will ignore requests of this directions
    (useful for load balancers)
  • All
    librdkafka.
    options. If a config option starts with rdkafka.<option>, <option> will be
    passed directly to librdkafka config, so you can use whatever config you need.
    Recommended options are:
    • "rdkafka.socket.max.fails":"3"
    • "rdkafka.socket.keepalive.enable":"true"
    • "delivery.report.only.error":"true"
Docker Pull Command
Owner
redborder
Source Repository

Comments (0)