Sign inSign up

nutthaphon/hadoop-slave2

By nutthaphon

•Updated about 7 years ago

This slave node is consist of datanode, nodemanager and so on. Use this image with hadoop-master

Image
0

118

nutthaphon/hadoop-slave2 repository overview

My Hadoop cluster is consist of :

  • Master node: hdmaster
  • Slave node: hdslave1
  • Slave node: hdslave2 (this image)

Add below parameters in docker-compose-cluster.yml

version: "3"

services:
  hdslave2:
    image: nutthaphon/hadoop-slave2:3.1.2-spark2.4.3-jdk8
    container_name: hdslave2
    hostname: hdslave2
    working_dir: "/spark"
    #user: "1000:1000"
    stdin_open: true
    tty: true
    ports:
      # SSH
      - "2222:22"
      # Hadoop Datanode Web
      - "9864:9864"
      # Spark Context Web UI
      - "4040:4040"
      # Node Manager
    volumes:
      - hdslave2-hadoop:/hadoop
      - java-program:/java
      - spark-program:/spark
  
volumes:
  java-program:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /home/nutt/Docker/bigdata/java/jdk1.8.0_211
  spark-program:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /home/nutt/Docker/bigdata/spark/spark-2.4.3-bin-hadoop2.7
  hdslave2-hadoop:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /home/nutt/Docker/bigdata/hadoop-3.1.2-cluster/hdslave2

Tag summary

Content type

Image

Digest

Size

1.2 GB

Last updated

about 7 years ago

docker pull nutthaphon/hadoop-slave2:3.1.2-spark2.4.3-jdk8