Sign inSign up

kentik/blueflow

By kentik

Updated over 5 years ago

This image sends flow logs from the IBM Cloud platform into Kentik.

Image
0

552

kentik/blueflow repository overview

This image sends flow logs from the IBM Cloud platform into Kentik.

To learn more about this, see:

Flags exposed by the binary:

Usage of /usr/bin/blueflow:
  -api_email string
    	Kentik Email Address
  -api_root string
    	API url prefix. If not set, defaults to https://api.kentik.com (default "https://api.kentik.com")
  -archive
    	If set, archive the flows with the prefix "processed" instead of deleting after processing
  -auth_endpoint string
    	Auth URL to auth IBM cloud on (default "https://iam.cloud.ibm.com/identity/token")
  -bucket_name string
    	Bucket to find VPC flow logs in.
  -dest string
    	Address to send flow to. If not set, defaults to https://flow.kentik.com (default "https://flow.kentik.com")
  -export_id int
    	Kentik export id
  -healthcheck string
    	Bind to this interface to allow healthchecks
  -log_level string
    	Logging Level (default "debug")
  -metalisten string
    	HTTP port to bind on (default "localhost:0")
  -metrics string
    	Metrics Configuration. none|syslog|stderr|graphite:127.0.0.1:2003 (default "syslog")
  -olly_dataset string
    	Olly dataset name
  -olly_write_key string
    	Olly dataset name
  -plan_id int
    	Kentik Plan ID to use for devices
  -regions string
    	Any comma seperated mix of us-south,us-east,eu-gb,eu-de,jp-tok (default "us-south")
  -redis string
    	host:port string on which to connect to redis
  -redis_db int
    	Redis DB to use
  -sample_rate int
    	Sampling rate to use. 1 -> 1:1 sampling, 2 -> 1:2 sampling and so on. (default 1)
  -service_endpoint string
    	Service URL to connect to IBM cloud on (default "s3.private.us-south.cloud-object-storage.appdomain.cloud")
  -service_name string
    	Service identifier (default "blueflow")
  -shard_follower
    	If true, this process is a sharding follower.
  -shard_leader
    	If true, this process is a sharding leader.
  -site_id int
    	Kentik Site ID to use for devices
  -stdout
    	Log to stdout (default true)
  -v	Show version and build information

This requires the following variables to be set:

  • IBM_VPC_API_KEY
  • IBM_COS_API_KEY
  • IBM_SERVICE_INSTANCE
  • ID of the COS which contains the bucket logs are written to.
  • IBM_BUCKET_NAME
  • Name of the log bucket.
  • IBM_ENDPOINT
  • Endpoint to use to access this bucket.
  • KENTIK_API_TOKEN
  • Token for Kentik's API.
  • KENTIK_EMAIL
  • Email address associated with this token.
  • KENTIK_PLAN_ID
  • Kentik billing plan ID to store flow under.
  • KENTIK_EXPORT_ID
  • Kentik cloud export ID to store flow under.

To run, a basic example is:

docker run             \
       -d              \
       --name blueflow \
       -p 8083:8083 \
       -e IBM_VPC_API_KEY=$IBM_VPC_API_KEY \
       -e KENTIK_API_TOKEN=$KENTIK_API_TOKEN \
       -e IBM_COS_API_KEY=$IBM_COS_API_KEY \
       -e IBM_SERVICE_INSTANCE=$IBM_SERVICE_INSTANCE \
       kentik/blueflow:v1                                             \
       --bucket_name=$IBM_BUCKET_NAME \
       --log_level=info          \
       --service_endpoint=$IBM_ENDPOINT \
       --api_email=$KENTIK_EMAIL \
       --plan_id=$KENTIK_PLAN_ID \
       --export_id=$KENTIK_EXPORT_ID

Verify that this is working and get some useful stats with

curl 127.0.0.1:8083/service/info | jq '.'

A sample docker-compose.yaml file for a more complex setup is:

version: '3'
services:
  redis:
    image: "redis"
    networks:
      blueflow:
        aliases:
          - redis
  blue_leader:
    image: "kentik/blueflow:v1"
    depends_on:
      - redis
    ports:
      - "8083"
    environment:
      - IBM_VPC_API_KEY=$IBM_VPC_API_KEY
      - KENTIK_API_TOKEN=$KENTIK_API_TOKEN
      - IBM_COS_API_KEY=$IBM_COS_API_KEY
      - IBM_SERVICE_INSTANCE=$IBM_SERVICE_INSTANCE
    entrypoint: ["/usr/bin/blueflow", "--bucket_name", "$IBM_BUCKET_NAME", "--log_level", "info", "--service_endpoint", "$IBM_ENDPOINT", "--api_email", "$KENTIK_EMAIL", "--plan_id", "$KENTIK_PLAN_ID", "--export_id", "$KENTIK_EXPORT_ID", "--redis", "redis:6379", "--shard_leader", "--metalisten", "0.0.0.0:8083"]
    networks:
      blueflow:
        aliases:
          - blue_leader
  blue_follower:
    image: "kentik/blueflow:v1"
    depends_on:
      - redis
      - blue_leader
    ports:
      - "8083"
    environment:
      - IBM_VPC_API_KEY=$IBM_VPC_API_KEY
      - KENTIK_API_TOKEN=$KENTIK_API_TOKEN
      - IBM_COS_API_KEY=$IBM_COS_API_KEY
      - IBM_SERVICE_INSTANCE=$IBM_SERVICE_INSTANCE
    entrypoint: ["/usr/bin/blueflow", "--bucket_name", "$IBM_BUCKET_NAME", "--log_level", "info", "--service_endpoint", "$IBM_ENDPOINT", "--api_email", "$KENTIK_EMAIL", "--plan_id", "$KENTIK_PLAN_ID", "--export_id", "$KENTIK_EXPORT_ID", "--redis", "redis:6379", "--shard_follower", "--metalisten", "0.0.0.0:8083", "--sample_rate", "2"]
    networks:
      blueflow:
        aliases:
          - blue_follower

networks:
  blueflow:

Tag summary

Content type

Image

Digest

Size

59.7 MB

Last updated

over 5 years ago

docker pull kentik/blueflow:v1