Public Repository

Last pushed: 11 days ago
Short Description
Apache Flink® is a powerful open-source distributed stream and batch processing framework.
Full Description

Supported tags and respective Dockerfile links

(amd64/flink build job)

Quick reference

What is Apache Flink?

Apache Flink is an open source stream processing framework with powerful stream- and batch-processing capabilities.

Learn more about Flink at https://flink.apache.org/

wikipedia.org/wiki/Apache_Flink

How to use this Docker image

Run a Flink local cluster

To run a single Flink local cluster:

$ docker run --name flink_local -p 8081:8081 -t amd64/flink local

Then with a web browser go to http://localhost:8081/ to see the Flink Web Dashboard (adjust the hostname for your Docker host).

To use Flink, you can submit a job to the cluster using the Web UI or you can also do it from a different Flink container, for example:

$ docker run --rm -t amd64/flink flink run -m <jobmanager:port> -c <your_class> <your_jar> <your_params>

Running a JobManager or a TaskManager

You can run a JobManager (master).

$ docker run --name flink_jobmanager -d -t amd64/flink jobmanager

You can also run a TaskManager (worker). Notice that workers need to register with the JobManager directly or via ZooKeeper so the master starts to send them tasks to execute.

$ docker run --name flink_taskmanager -d -t amd64/flink taskmanager

Running a cluster using Docker Compose

With Docker Compose you can create a Flink cluster:

version: "2.1"
services:
  jobmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6123"
    ports:
      - "8081:8081"
    command: jobmanager
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

  taskmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6121"
      - "6122"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

and just run docker-compose up.

Scale the cluster up or down to N TaskManagers:

docker-compose scale taskmanager=<N>

Configuration

These are the default ports used by the Flink image:

  • The Web Client is on port 8081
  • JobManager RPC port 6123
  • TaskManagers RPC port 6122
  • TaskManagers Data port 6121

About this repository

This repository is available on github.com/docker-flink/docker-flink, and the official build is on the Docker Hub.

This repository is maintained by members of the Apache Flink community.

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

      https://www.apache.org/licenses/LICENSE-2.0

Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.

Docker Pull Command
Owner
amd64

Comments (0)