Sign inSign up

hivehub/synap

By hivehub

•Updated 1 day ago

Synap is a modern, production-ready in-memory data platform built in Rust that provides

Image
Machine learning & AI
Developer tools
1

2.2K

hivehub/synap repository overview

⁠Synap - High-Performance In-Memory Data Platform

Docker Pulls Docker Image Size License

Unified data platform combining Redis, RabbitMQ, and Kafka in a single high-performance system

Synap is a modern, production-ready in-memory data platform built in Rust that provides:

  • 💾 Key-Value Store - Redis-compatible operations (SET/GET/INCR, TTL/PX, MULTI/EXEC, SCAN cursors, LFU/LRU eviction, zero-copy reads, integer-encoded counters)
  • 📨 Message Queues - RabbitMQ-style queues with ACK/NACK guarantees
  • 📡 Event Streams - Kafka-style partitioned streams with consumer groups
  • 🔔 Pub/Sub - Topic-based messaging with wildcards + Redis-style keyspace notifications (notify-keyspace-events)
  • 👁️ KV Watch - kv.watch() streams the new value on every mutation, so a watcher never has to re-GET; over SynapRPC or the /kv/ws WebSocket
  • 🧱 Collections - Hashes, Lists, Sets, Sorted Sets (blocking pops BLPOP/BZPOPMIN, HSCAN/SSCAN/ZSCAN, packed small-collection encodings), Bitmaps, HyperLogLog, Geospatial
  • 🔐 Authentication - Users, API keys, per-command ACLs
  • 💪 Persistence - WAL (group commit) + snapshots covering every datatype
  • 🔄 Replication - Master-replica with safe mid-stream replica joins
⁠⚡ Performance (v1.0, measured vs Redis 7.4)

Same redis-benchmark harness against both servers in Docker (-r 1000000 -c 50 -P 16, median-of-3 — see docs/benchmarks/redis-vs-synap.md⁠):

WorkloadSynap vs Redis 7
Multi-key SET / INCR / LPUSH1.05–1.22× faster
Multi-key GET / LPOPparity (0.91–0.96)
High concurrency (c=200) SET / GET / INCR1.03–1.19× faster
Single hot key (Redis's best case)0.85–0.95 (parity band)

Synap's 64-way sharded, multi-core architecture pulls ahead exactly where real workloads live: many keys, many connections.

⁠🚀 Quick Start

⁠Basic Usage
# Pull the image
docker pull hivehub/synap:latest

# Run a simple instance (all three protocols)
docker run -d \
  --name synap \
  -p 15500:15500 \
  -p 15501:15501 \
  -p 6379:6379 \
  -v synap-data:/data \
  hivehub/synap:latest

# Check health
curl http://localhost:15500/health
⁠With Docker Compose
version: '3.8'

services:
  synap:
    image: hivehub/synap:latest
    container_name: synap-server
    ports:
      - "15500:15500"  # HTTP/REST API + StreamableHTTP
      - "15501:15501"  # SynapRPC binary protocol (synap://)
      - "6379:6379"    # RESP3 Redis-compatible (resp3://)
    volumes:
      - synap-data:/data
      - ./config.yml:/app/config.yml:ro
    restart: unless-stopped
    # No healthcheck block needed — the image ships its own HEALTHCHECK. Do not
    # override it with a wget/curl probe: the image is built `FROM scratch` and
    # has no shell and no wget, so such a probe can only ever report unhealthy.

volumes:
  synap-data:

⁠📋 Supported Tags

  • latest - Latest stable release
  • 1.2.0 - KV watch, and the SynapRPC binary transport moved onto Thunder⁠ on both ends of the wire. Wire v1 is unchanged — a pre-1.2.0 client keeps working against a 1.2.0 server
  • 1.0.0 - The 1.0 release (crates workspace, security/durability audit, Redis-parity performance)
  • 0.13.0 / 0.12.0 - Previous releases

latest, 1.2.0 and 1.0.0 are multi-arch manifest lists — a single docker pull resolves the native image for your platform:

  • linux/amd64 - Intel/AMD 64-bit (AVX2 SIMD runtime-detected)
  • linux/arm64 - ARM 64-bit: Apple Silicon, AWS Graviton, Raspberry Pi (NEON SIMD runtime-detected)

Tags 0.13.0 and older are linux/amd64 only.

⁠🔧 Configuration

⁠Environment Variables
VariableDescriptionDefault
SYNAP_SERVER_ADDRServer bind address0.0.0.0:15500
SYNAP_REPLICATION_ADDRReplication bind address0.0.0.0:15501
SYNAP_DATA_DIRData directory path/data
SYNAP_CONFIG_PATHConfig file path/app/config.yml
SYNAP_AUTH_ENABLEDEnable authenticationfalse
SYNAP_AUTH_REQUIRE_AUTHRequire authentication for all requestsfalse
SYNAP_AUTH_ROOT_USERNAMERoot user usernameroot
SYNAP_AUTH_ROOT_PASSWORDRoot user password(must be set)
SYNAP_AUTH_ROOT_ENABLEDEnable root usertrue
⁠Volume Mounts
PathDescriptionRequired
/dataData directory (WAL, snapshots)Yes
/app/config.ymlCustom configuration fileNo
⁠Ports
PortProtocolDescription
15500HTTP/TCPREST API, StreamableHTTP, WebSocket
15501TCPSynapRPC binary protocol (synap://host:15501)
6379TCPRESP3 Redis-compatible protocol (resp3://host:6379)
15600TCPReplication (master-slave, internal)

⁠📖 Usage Examples

⁠Connecting via Different Protocols

Synap exposes three client-facing protocols. Choose based on your use case:

# HTTP/REST — universal, any HTTP client
curl http://localhost:15500/kv/get/mykey

# SynapRPC — lowest latency, use Synap SDKs (Rust, TypeScript, Python, etc.)
# Rust SDK:
#   let client = SynapClient::new(SynapConfig::new("synap://localhost:15501"))?;

# RESP3 — Redis-compatible, any Redis client
redis-cli -p 6379 GET mykey
⁠Basic Key-Value Operations
# Set a key
curl -X POST http://localhost:15500/kv/set \
  -H "Content-Type: application/json" \
  -d '{"key": "user:1", "value": "John Doe", "ttl": 3600}'

# Get a key
curl http://localhost:15500/kv/get/user:1

# Delete a key
curl -X DELETE http://localhost:15500/kv/delete/user:1
⁠Queue Operations
# Publish a message
curl -X POST http://localhost:15500/queue/publish \
  -H "Content-Type: application/json" \
  -d '{
    "queue": "tasks",
    "message": "process-video",
    "priority": 1
  }'

# Consume a message
curl http://localhost:15500/queue/consume/tasks

# Acknowledge a message
curl -X POST http://localhost:15500/queue/ack \
  -H "Content-Type: application/json" \
  -d '{"queue": "tasks", "message_id": "msg-123"}'
⁠Event Stream Operations
# Publish an event
curl -X POST http://localhost:15500/stream/publish \
  -H "Content-Type: application/json" \
  -d '{
    "room": "chat-room-1",
    "event": "message",
    "data": {"text": "Hello, World!"}
  }'

# Consume events
curl http://localhost:15500/stream/consume/chat-room-1?offset=0&limit=10
⁠Pub/Sub Operations
# Publish to a topic
curl -X POST http://localhost:15500/pubsub/publish \
  -H "Content-Type: application/json" \
  -d '{
    "topic": "notifications.email",
    "message": "New order received"
  }'

⁠🔐 Authentication Setup

⁠Enable Authentication
docker run -d \
  --name synap \
  -p 15500:15500 \
  -p 15501:15501 \
  -p 6379:6379 \
  -v synap-data:/data \
  -e SYNAP_AUTH_ENABLED=true \
  -e SYNAP_AUTH_REQUIRE_AUTH=true \
  -e SYNAP_AUTH_ROOT_USERNAME=admin \
  -e SYNAP_AUTH_ROOT_PASSWORD=SecurePassword123! \
  -e SYNAP_AUTH_ROOT_ENABLED=true \
  hivehub/synap:latest
⁠Using Authentication
# Basic Auth
curl -X POST http://localhost:15500/kv/set \
  -u admin:SecurePassword123! \
  -H "Content-Type: application/json" \
  -d '{"key": "test", "value": "data"}'

# API Key (Bearer Token)
curl -X POST http://localhost:15500/kv/set \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"key": "test", "value": "data"}'

⁠🔄 Replication Setup

⁠Master Node
version: '3.8'

services:
  synap-master:
    image: hivehub/synap:latest
    ports:
      - "15500:15500"  # HTTP/REST
      - "15501:15501"  # SynapRPC
      - "6379:6379"    # RESP3
      - "15600:15600"  # Replication (internal)
    volumes:
      - master-data:/data
      - ./config-master.yml:/app/config.yml:ro
    restart: unless-stopped

volumes:
  master-data:
⁠Replica Node
version: '3.8'

services:
  synap-replica:
    image: hivehub/synap:latest
    ports:
      - "15510:15500"  # HTTP/REST
      - "15511:15501"  # SynapRPC
      - "6380:6379"    # RESP3
    volumes:
      - replica-data:/data
      - ./config-replica.yml:/app/config.yml:ro
    environment:
      - SYNAP_MASTER_ADDRESS=master-host:15600
    restart: unless-stopped
    depends_on:
      - synap-master

volumes:
  replica-data:

⁠📊 Monitoring

⁠Health Check
# Check server health
curl http://localhost:15500/health

# Response: {"service":"synap","status":"healthy","version":"1.2.0"}
⁠Prometheus Metrics
# Expose metrics endpoint (if enabled in config)
curl http://localhost:15500/metrics
⁠Server Information
# Get server stats
curl http://localhost:15500/info

# Get KV store stats
curl http://localhost:15500/kv/stats

# Get queue stats
curl http://localhost:15500/queue/stats/queue-name

⁠🛠️ Advanced Configuration

⁠Custom Configuration File
# Create custom config.yml
cat > config.yml <<EOF
server:
  host: "0.0.0.0"
  port: 15500

kv_store:
  max_memory_mb: 1024
  eviction_policy: "lru"

queue:
  enabled: true
  max_depth: 100000

# Binary protocols (enabled by default)
synap_rpc:
  enabled: true
  host: "0.0.0.0"
  port: 15501

resp3:
  enabled: true
  host: "0.0.0.0"
  port: 6379

persistence:
  enabled: true
  wal:
    enabled: true
    path: "/data/wal/synap.wal"
  snapshot:
    enabled: true
    directory: "/data/snapshots"
    interval_secs: 300
EOF

# Run with custom config
docker run -d \
  --name synap \
  -p 15500:15500 \
  -p 15501:15501 \
  -p 6379:6379 \
  -v synap-data:/data \
  -v $(pwd)/config.yml:/app/config.yml:ro \
  hivehub/synap:latest
⁠Production Deployment
version: '3.8'

services:
  synap:
    image: hivehub/synap:1.2.0
    container_name: synap-production
    ports:
      - "15500:15500"  # HTTP/REST
      - "15501:15501"  # SynapRPC
      - "6379:6379"    # RESP3
    volumes:
      - /var/lib/synap/data:/data
      - /etc/synap/config.yml:/app/config.yml:ro
    environment:
      - SYNAP_SERVER_ADDR=0.0.0.0:15500
      - SYNAP_AUTH_ENABLED=true
      - SYNAP_AUTH_REQUIRE_AUTH=true
      - SYNAP_AUTH_ROOT_USERNAME=${SYNAP_ROOT_USER}
      - SYNAP_AUTH_ROOT_PASSWORD=${SYNAP_ROOT_PASS}
    restart: always
    # The image's built-in HEALTHCHECK already probes /health every 30s using the
    # server binary itself. Override it only with `synap-server --health-check`,
    # never with wget or curl — neither exists in a `scratch` image.
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 2G
        reservations:
          cpus: '1'
          memory: 1G

⁠🔍 Troubleshooting

⁠Container Won't Start
# Check logs
docker logs synap

# Check if port is already in use
netstat -tuln | grep 15500

# Verify data directory permissions
ls -la /var/lib/synap/data
⁠Performance Issues
# Check memory usage
docker stats synap

# Check disk I/O
docker exec synap df -h /data

# Monitor slow queries (if enabled)
curl http://localhost:15500/slowlog
⁠Replication Issues
# Check replication status
curl http://localhost:15500/info | jq .replication

# Check replica connection
docker logs synap-replica | grep replication

# Verify network connectivity
docker exec synap-replica ping synap-master

⁠📚 Additional Resources

⁠🤝 Support

⁠📄 License

This project is licensed under the Apache License 2.0 - see the LICENSE⁠ file for details.

⁠🏷️ Image Details

  • Base Image: scratch — the fully static musl binary ships with no distro at all (no shell, no package manager, zero distro CVE surface; Docker Scout: 0C/0H/0M/0L)
  • Size: ~19MB — static binary + CA bundle + config
  • Health check: built into the server binary (synap-server --health-check) — no wget/shell required
  • Architecture: Multi-arch (amd64, arm64)
  • User: Non-root (synap:synap, UID 1000)
  • Health Check: Built-in HTTP health endpoint
  • Supply chain: SBOM + provenance (SLSA) attestations published alongside every tag — built with docker buildx --sbom=true --provenance=mode=max. Verify with docker buildx imagetools inspect hivehub/synap:1.2.0 --format "{{ json .SBOM }}".

Maintained by: HiveLLM⁠

Image Registry: Docker Hub⁠

Tag summary

Content type

Image

Digest

sha256:58d898997…

Size

8.9 MB

Last updated

1 day ago

docker pull hivehub/synap