Public | Automated Build

Last pushed: 2 years ago
Short Description
Hadoop with HIVE and HBASE cluster.
Full Description

We are the cloud computing research team. We're trying to do something for the Cloud-Relative Starters.

This release is FOR Hadoop Cluster QuickStarters.

Hadoop 2.5.2 Only. With HIVE 1.2.1

This release is very the same as the other project https://hub.docker.com/r/h2comm/docker-hadoop-cluster/.

You can use our Dockerfile building your own images, or use docker poll directly.
We highly recommend the former.

Configuration

In whatever ways, before you start, you have to do some configurations as follows:

  1. Assuming your namenode info like this "namenode 10.0.0.111", modify core-site.xml with your IP, default port is 9000;
  2. Do the same work to hdfs-site.xml,yarn-site.xml and mapred-site.xml, modify replication property if you need;
  3. Modify hosts file, add ALL the host names and IPs. With default configuration we have 1 namenode and 2datanode;
  4. Add all hostnames or IPs to the file slave.
  5. Search hive.exec.scratchdir in hive-site.xml and set to your namenode, also the hive.exec.scratchdir, hive.metastore.warehouse.dir, and hive.metastore.uris.

    javax.jdo.option.ConnectionURL set to your mysql-server.

##RUN

OK, well, almost done now.
After steps above, you can start building image.
Then start containers like this:

>docker run -itd --name datanode1 -h datanode1 --net=none YOUR.IMAGE.NAME /etc/bootstrap.sh -d

>pipework br0 datanode1 10.0.1.11/16

>docker run -itd --name datanode2 -h datanode2 --net=none YOUR.IMAGE.NAME /etc/bootstrap.sh -d

>pipework br0 datanode2 10.0.1.12/16

>sleep 10

>docker run -itd --name namenode -h namenode --net=none YOUR.IMAGE.NAME /etc/bootstrap.sh -dmaster

>pipework br0 namenode 10.0.0.111/16

The HIVE is not auto start(easy to debug, -_-), you can modify the bootstrap.sh yourself(try hive -h).
OR type >hive --service metastore to go .

By using docker logs -f namenode you can see these:

Container's IP
Hosts
HDFS report

Congras!

Now you get a cluster with one namenode and two datanodes. Type "hive" on namenode, enjoy it!

Contact us : Zhongliang xiaozhongliang@h2comm.com.cn .

Docker Pull Command
Owner
h2comm

Comments (0)