Sign inSign up

badele/debian-hadoop

By badele

Updated almost 11 years ago

A hadoop cluster with 3 datanodes and 1 namenode

Image
0

10K+

badele/debian-hadoop repository overview

A hadoop cluster with 3 datanodes and 1 namenode

A hadoop cluster for learn the hadoop configuration.

How to use

Init HDFS

Run this only one time

docker-compose run namenode hdfs namenode -format
Start hadoop cluster

Launching the namenode and 3 datanodes with below command

docker-compose up -d
Put data in the HDFS
hadoop fs -put local_filename hdfs://localhost/
docker-compose.yml
namenode:
  build: .
  hostname: namenode
  volumes:
    - /docker_shared/debian-hadoop/namenode:/shared
  ports:
    - "8020:8020"
    - "50070:50070"
  command: hdfs namenode


datanode1:
  build: .
  hostname: datanode1
  volumes:
      - /docker_shared/debian-hadoop/datanode1:/shared
  links:
      - namenode
  ports:
      - "5001:50075"
  command: hdfs datanode

datanode2:
  build: .
  hostname: datanode2
  volumes:
      - /docker_shared/debian-hadoop/datanode2:/shared
  links:
      - namenode
  ports:
      - "5002:50075"
  command: hdfs datanode

datanode3:
  build: .
  hostname: datanode3
  volumes:
      - /docker_shared/debian-hadoop/datanode3:/shared
  links:
      - namenode
  ports:
      - "5003:50075"
  command: hdfs datanode

Tag summary

Content type

Image

Digest

sha256:259f6dcbc

Size

443.1 MB

Last updated

over 11 years ago

docker pull badele/debian-hadoop