Public | Automated Build

Last pushed: 2 years ago
Short Description
Hortonworks Data Platform for automated install with Ambari (HDP 2.3.4)
Full Description

Hortonworks Data Platform

Automated Install with Ambari

  • Launch the container running Ambari Server and Ambari Agent
    docker run  -p 8080:8080 --rm -ti --privileged --name hdp -h hdp torusware/hortondataplatform-examples
    
  • You can also deploy a cluster automatically using Ambari blueprints
    docker run -e BLUEPRINT_URL=<ambari blueprint url> -e CLUSTER_TEMPLATE_URL=<cluster template url> -p 8080:8080 --rm -ti --privileged -h hdp  torusware/hortondataplatform-examples
    
  • Open your browser and go to http://localhost:8080
  • Access a running container through ssh
    $ sshpass -p 'torus' ssh root@<container IP>
    

Spark examples

Configure environment

  • Set up the Hadoop ecosystem

    1. Pull container docker pull torusware/hortondataplatform-examples
    2. Launch container as above: docker run -p 8080:8080 --rm -ti --privileged --name hdp -h hdp torusware/hortondataplatform-examples
    3. Access http://localhost:8080 from your host browser
    4. Login: admin/admin
    5. Launch install wizard with the hostname and ssh key set in the configuration step
    6. Install services: HDFS, YARN, Ambari Metrics and Spark
  • Run examples within the container shell

Run examples

  • TwitterPopularTags

    spark-submit --class org.apache.spark.examples.streaming.TwitterPopularTags \           
    /usr/hdp/current/spark-client/lib/spark-examples-1.5.2.2.3.4.0-3485-hadoop2.7.1.2.3.4.0-3485.jar \
    <consumer key> <consumer secret> <access token> <access token secret>  [<filters>]
    
Docker Pull Command
Owner
torusware