databuffhub/databuff-proxy

•Updated about 2 months ago

Fans out SkyWalking Agent traffic to multiple backends in parallel.

Image

185

databuffhub/databuff-proxy repository overview

⁠databuff-proxy

Fans out SkyWalking Agent traffic to multiple backends in parallel. If one backend dies or is turned off, the others keep working.

SkyWalking Agent ──► databuff-proxy ──► Backend A
                                ├──► Backend B
                                └──► Backend C …

Multi-arch image: linux/amd64 + linux/arm64. docker pull auto-selects the right arch.

⁠Two ways to configure

You can run databuff-proxy in one of two modes — pick whichever fits your workflow:

	Method 1: Environment variables	Method 2: Mounted config file
Best for	Containers, one-shot deploys, CI	Persistent toggles, full tuning, existing config
Config source	`-e` flags	A `config.yaml` mounted into the container
Admin-UI write toggles	In-memory only (reset on restart)	Persisted back to the file
Needs a volume	No	Yes (writable)

⁠Method 1 — Environment variables (no config file)

docker run -d --name databuff-proxy \
  -p 11800:11800 \
  -p 9090:9090 \
  -e BACKENDS="skywalking=192.168.50.114:11800,databuff=192.168.50.113:11800" \
  databuffhub/databuff-proxy:v0.1.0

11800: where the SkyWalking Agent connects (gRPC)
9090: admin UI at http://127.0.0.1:9090/
BACKENDS: forwarding targets, format name=addr[,name=addr], all enabled by default; disable one with :false

Point your SkyWalking Agent at the proxy address (default your-host:11800) instead of the backend, then restart the agent.

⁠Backends (required, at least one)

Two formats, pick one:

A. BACKENDS string (recommended)

BACKENDS="skywalking=192.168.50.114:11800,databuff=192.168.50.113:11800"
# disable one:
BACKENDS="skywalking=192.168.50.114:11800:true,databuff=192.168.50.113:11800:false"

B. BACKEND_<n>_* individual entries

BACKEND_1_NAME=skywalking
BACKEND_1_ADDR=192.168.50.114:11800
BACKEND_2_NAME=databuff
BACKEND_2_ADDR=192.168.50.113:11800
# BACKEND_2_ENABLED defaults to true if unset

⁠Listener / admin / logging

Variable	Default	Description
`LISTEN_GRPC_ADDR`	`:11800`	Where the SkyWalking Agent connects
`LISTEN_MAX_RECV_MSG_SIZE`	`16777216`	Max gRPC message size (bytes)
`ADMIN_ADDR`	`:9090`	Admin UI address
`ADMIN_TOKEN`	empty	If set, toggling write requires this token
`LOGGING_LEVEL`	`info`	`debug` / `info` / `warn` / `error`
`STATS_WINDOW_MINUTES`	`30`	Admin UI traffic window (minutes)

⁠Forwarding tuning (when dropping data / sending too slow)

Variable	Default	Description
`QUEUE_SIZE`	`4096`	Per-backend queue capacity
`WORKERS`	`2`	Concurrent senders per backend
`RPC_TIMEOUT`	`3s`	Timeout sending to backend
`DIAL_TIMEOUT`	`5s`	Connection timeout
`CIRCUIT_FAILURE_THRESHOLD`	`20`	Consecutive failures before tripping circuit
`CIRCUIT_OPEN_TIMEOUT`	`30s`	How long the circuit stays open before half-open probe

Durations use Go time.Duration format, e.g. 3s / 500ms / 2m.

Note: in env-var mode, admin-UI write toggles are in-memory only and reset on restart. To persist toggles, use Method 2.

⁠Method 2 — Mounted config file

Mount a writable config.yaml and pass its path with -config. The admin UI then persists write toggles back to the file; env vars still work and override the file.

# 1. prepare a config (example shipped in the repo)
cp configs/config.example.yaml ./config.yaml
# edit backends[].addr to point at your downstreams

# 2. run with the file mounted
docker run -d --name databuff-proxy \
  -p 11800:11800 \
  -p 9090:9090 \
  -v "$PWD/config.yaml:/etc/databuff-proxy/config.yaml" \
  databuffhub/databuff-proxy:v0.1.0 \
  -config /etc/databuff-proxy/config.yaml

config.yaml example:

listen:
  grpc_addr: ":11800"
  max_recv_msg_size: 16777216

backends:
  - name: skywalking
    addr: "192.168.50.114:11800"
    enabled: true
  - name: databuff
    addr: "192.168.50.113:11800"
    enabled: true

path_defaults:
  queue_size: 4096
  workers: 8
  rpc_timeout: 3s
  dial_timeout: 5s
  shutdown_drain_timeout: 10s
  circuit:
    failure_threshold: 50
    open_timeout: 60s
    half_open_max: 2

admin:
  addr: ":9090"
  # token: ""

stats:
  window_minutes: 30
  bucket_interval: 1m

logging:
  level: info

The mounted file must be writable — the admin UI rewrites it when you toggle a backend's write switch. Read-only mounts break persistence.

You can also combine both: mount a file for persistence and override individual fields with -e. Env vars take precedence over the file.

⁠Docker Compose

Method 1 (env vars):

services:
  databuff-proxy:
    image: databuffhub/databuff-proxy:v0.1.0
    container_name: databuff-proxy
    restart: unless-stopped
    ports:
      - "11800:11800"
      - "9090:9090"
    environment:
      BACKENDS: "skywalking=192.168.50.114:11800,databuff=192.168.50.113:11800"

Method 2 (config file):

services:
  databuff-proxy:
    image: databuffhub/databuff-proxy:v0.1.0
    container_name: databuff-proxy
    restart: unless-stopped
    ports:
      - "11800:11800"
      - "9090:9090"
    volumes:
      - ./config.yaml:/etc/databuff-proxy/config.yaml
    command: ["-config", "/etc/databuff-proxy/config.yaml"]

docker compose up -d

⁠What the admin UI can do

Action	Description
Toggle write per backend	Click the "write" column; takes effect immediately
View last 30 min of traffic	Success, failure, queue drops, circuit trips, etc.
See if a backend is tripped	"circuit" column, auto-detected

⁠Health check

The image ships a /healthcheck binary; the default container HEALTHCHECK probes 127.0.0.1:9090/healthz:

curl http://127.0.0.1:9090/healthz
# ok

⁠More

Source & full docs: https://github.com/databufflabs/databuff-proxy⁠
Configuration & tuning: https://github.com/databufflabs/databuff-proxy/blob/main/CONFIG.md⁠
Build & packaging: https://github.com/databufflabs/databuff-proxy/blob/main/BUILD.md⁠

Tag summary

Recent tags

Content type

Image

Digest

sha256:0ede5419c…

Size

8.1 MB

Last updated

about 2 months ago

docker pull databuffhub/databuff-proxy:v0.1.0