wangdongqing/hadoop-hbase repository overview

There is a sample compose yaml at hadoop-cluster-docker/Hadoop/start-hadoop-hbase-zk_staticIP.yaml. This sample composes:

1. Zookeeper cluster composed by 3 containers
2. Four Hadoop containers which are:
   • hadoopmaster as namenode, resourcemanager and hmaster(on startup)
   • hadoopstandby as secondarynamenode
   • hadoopslave1 & hadoopslave2 as datanode, nodemanager and regionserver(on startup)

For connivence, static IP is used so that /etc/hosts could be written before containers are up.

The reason of not using dynamically assigned IP is that /etc/hosts is readonly when containers are up so HBase will rely on Docker DNS(127.0.0.11) to get region servers. But for each container, both hostname and hostname.network are available by DNS, which causes each region server being regarded as two that is prone to get exceptions.

Alternative solution is to use dnsmasq to mask hostname.network. May implement in future.

How to run: docker-compose -f start-hadoop-hbase-zk_staticIP.yaml up -d

Then you can check from Host of Docker with: localhost:50070 localhost:8088 localhost:16010

A convenient approach to HBase shell is: docker exec -it hadoopstandby bash->hbase shell