zilutian/graph-analytics repository overview

docker create --name data cloudsuite/twitter-dataset-graph
docker network create spark-net 
docker run -dP --net spark-net --hostname spark-master --name spark-master --cpuset-cpus=0-5 zilutian/spark master
docker run -dP --net spark-net --volumes-from data --name spark-worker-01 --cpuset-cpus=6-11 zilutian/spark worker spark://spark-master:7077
docker run -dP --net spark-net --volumes-from data --name spark-worker-02 --cpuset-cpus=12-17 zilutian/spark worker spark://spark-master:7077
docker run -dP --net spark-net --volumes-from data --name spark-worker-03 --cpuset-cpus=18-23 zilutian/spark worker spark://spark-master:7077
docker run --net spark-net --volumes-from data zilutian/graph-analytics:amd64 \
--driver-memory 1g --executor-memory 4g --master spark://spark-master:7077

...
19/05/01 15:41:41 INFO TaskSetManager: Starting task 10.0 in stage 97.0 (TID 889, 172.31.0.4, executor 1, partition 10, PROCESS_LOCAL, 8370 bytes)
19/05/01 15:41:41 INFO TaskSetManager: Starting task 9.0 in stage 97.0 (TID 890, 172.31.0.3, executor 0, partition 9, PROCESS_LOCAL, 8370 bytes)
19/05/01 15:41:41 INFO TaskSetManager: Starting task 11.0 in stage 97.0 (TID 891, 172.31.0.5, executor 2, partition 11, PROCESS_LOCAL, 8370 bytes)
...
19/05/01 15:41:42 INFO TaskSchedulerImpl: Removed TaskSet 97.0, whose tasks have all completed, from pool 
19/05/01 15:41:42 INFO DAGScheduler: ResultStage 97 (sum at GraphAnalytics.scala:37) finished in 0.471 s
19/05/01 15:41:42 INFO DAGScheduler: Job 8 finished: sum at GraphAnalytics.scala:37, took 0.476627 s
Total PageRank = 1.1316811000021858E7
Running time = 202367
...