Official Repository

Last pushed: 6 days ago
Short Description
Apache Flink® is a powerful open-source distributed stream and batch processing framework.
Full Description

Supported tags and respective Dockerfile links

Quick reference

What is Apache Flink?

Apache Flink is an open source stream processing framework with powerful stream- and batch-processing capabilities.

Learn more about Flink at https://flink.apache.org/

wikipedia.org/wiki/Apache_Flink

How to use this Docker image

Run a Flink local cluster

To run a single Flink local cluster:

$ docker run --name flink_local -p 8081:8081 -t flink local

Then with a web browser go to http://localhost:8081/ to see the Flink Web Dashboard (adjust the hostname for your Docker host).

To use Flink, you can submit a job to the cluster using the Web UI or you can also do it from a different Flink container, for example:

$ docker run --rm -t flink flink run -m <jobmanager:port> -c <your_class> <your_jar> <your_params>

Running a JobManager or a TaskManager

You can run a JobManager (master).

$ docker run --name flink_jobmanager -d -t flink taskmanager

You can also run a TaskManager (worker). Notice that workers need to register with the JobManager directly or via ZooKeeper so the master starts to send them tasks to execute.

$ docker run --name flink_taskmanager -d -t flink taskmanager

Running a cluster using Docker Compose

With Docker Compose you can create a Flink cluster:

version: "2.1"
services:
  jobmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6123"
    ports:
      - "8081:8081"
    command: jobmanager
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

  taskmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6121"
      - "6122"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

and just run docker-compose up.

Scale the cluster up or down to N TaskManagers:

docker-compose scale taskmanager=<N>

Configuration

These are the default ports used by the Flink image:

  • The Web Client is on port 8081
  • JobManager RPC port 6123
  • TaskManagers RPC port 6122
  • TaskManagers Data port 6121

About this repository

This repository is available on github.com/docker-flink/docker-flink, and the official build is on the Docker Hub.

This repository is maintained by members of the Apache Flink community.

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

      https://www.apache.org/licenses/LICENSE-2.0

Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.

Docker Pull Command

Comments (0)