For full description, please refer to https://github.com/parsa-epfl/cloudsuite/blob/master/docs/benchmarks/in-memory-analytics.md
Regarding the dataset:
Sample commands for running the benchmark for multi-node setup: (Please adjust the cpu core assignment and image names accordingly)
docker create --name data cloudsuite/movielens-dataset
docker network create spark-net
docker run -dP --net spark-net --hostname spark-master --name spark-master --cpuset-cpus=0-5 zilutian/spark master
docker run -dP --net spark-net --volumes-from data --name spark-worker-01 --cpuset-cpus=6-11 zilutian/spark worker spark://spark-master:7077
docker run -dP --net spark-net --volumes-from data --name spark-worker-02 --cpuset-cpus=12-17 zilutian/spark worker spark://spark-master:7077
docker run -dP --net spark-net --volumes-from data --name spark-worker-03 --cpuset-cpus=18-23 zilutian/spark worker spark://spark-master:7077
docker run --net spark-net --volumes-from data zilutian/in-memory-analytics:amd64 /data/ml-latest-small /data/myratings.csv --master spark://spark-master:7077
Expected output for dataset latest-small:
Got 100004 ratings from 671 users on 9066 movies.
Training: 59876, validation: 20285, test: 19854
RMSE (validation) = 0.9282809816943224 for the model trained with rank = 8, lambda = 0.1, and numIter = 10.
...
The best model was trained with rank = 12 and lambda = 0.1, and numIter = 10, and its RMSE on the test set is 0.9386957653824353.
The best model improves the baseline by 11.47%.
Movies recommended for you:
1: Her (2013)
...
Content type
Image
Digest
Size
287 MB
Last updated
over 7 years ago
docker pull zilutian/in-memory-analytics:arm64