Sign inSign up

mohammadzainabbas/graphstorm

By mohammadzainabbas

•Updated almost 3 years ago

AWS GraphStorm on CPU 🚀

Image
0

133

mohammadzainabbas/graphstorm repository overview

⁠GraphStorm on CPU (even on Apple silicon) 🚀

GraphStorm⁠ is a graph machine learning (GML) framework designed for enterprise use cases. It simplifies the development, training and deployment of GML models on industry-scale graphs (measured in billons of nodes and edges) by providing scalable training and inference pipelines of GML models.

Original repository for GraphStorm can be find here⁠. You can find more about GraphStorm with this blog article⁠ on AWS Machine Learning Blog.

This repository is for those who want to try GraphStorm on their local machine (with CPU).

⁠How to use GraphStorm on CPU ?
⁠1. Download the docker image
docker pull mohammadzainabbas/graphstorm:local
⁠2. Create an empty folder for volume
mkdir -p <path-to-local-folder-for-volume-mount>

For e.g:

mkdir -p ~/Desktop/graphstorm/docker_volume
⁠3. Create a docker container image
docker run --network=host -v <path-to-local-folder-for-volume-mount>:/dev/shm -d --name <container-name> mohammadzainabbas/graphstorm:local

For e.g:

docker run --network=host -v ~/Desktop/graphstorm/docker_volume:/dev/shm -d --name test mohammadzainabbas/graphstorm:local
⁠4. Connect to the container
docker container exec -it <container-name> /bin/bash

For e.g:

docker container exec -it test /bin/bash

Note: You can use VS Code's Remote Explorer extension as well to open VS Code inside the container

⁠About this repository

We took Dockerfile⁠ from GraphStorm repository⁠ and modified it as shown below:

FROM nvidia/cuda:11.3.1-devel-ubuntu20.04
ENV DEBIAN_FRONTEND=noninteractive
ENV HOME=/root

RUN apt-get update
RUN apt-get install -y --no-install-recommends build-essential python3-dev make g++ vim net-tools

RUN apt-get install -y python3-pip git wget psmisc
RUN apt-get install -y cmake

# Install Pytorch
RUN pip3 install torch --index-url https://download.pytorch.org/whl/cpu
# Install DGL
RUN pip3 install dgl -f https://data.dgl.ai/wheels/repo.html
# Install related Python packages
RUN pip3 install ogb==1.3.6 scipy pyarrow boto3 scikit-learn transformers

# Install other dependencies
RUN apt-get install -y cython3 libicu-dev
RUN pip3 install h5py psutil

RUN apt-get install -y unzip

# Download DGL source code
RUN cd /root; git clone --recursive https://github.com/dmlc/dgl.git

# Install GraphStorm from source code
RUN mkdir -p /graphstorm
COPY code/python/graphstorm /graphstorm/python/graphstorm
ENV PYTHONPATH="/graphstorm/python/:${PYTHONPATH}"

# Copy GraphStorm scripts and tools
COPY code/examples /graphstorm/examples
COPY code/inference_scripts /graphstorm/inference_scripts
COPY code/tools /graphstorm/tools
COPY code/training_scripts /graphstorm/training_scripts

# Set up SSH
RUN apt-get install -y openssh-client openssh-server
ENV SSH_PORT=2222
RUN cat /etc/ssh/sshd_config > /tmp/sshd_config && \
    sed "0,/^#Port 22/s//Port ${SSH_PORT}/" /tmp/sshd_config > /etc/ssh/sshd_config
ENV SSHDIR $HOME/.ssh
RUN mkdir -p ${SSHDIR}
RUN ssh-keygen -t rsa -f ${SSHDIR}/id_rsa -N ''
RUN cp ${SSHDIR}/id_rsa.pub ${SSHDIR}/authorized_keys

EXPOSE 2222
RUN mkdir /run/sshd
CMD ["/usr/sbin/sshd", "-D"]

Happy coding 🚀

Tag summary

Content type

Image

Digest

sha256:e47d90104…

Size

3.5 GB

Last updated

almost 3 years ago

docker pull mohammadzainabbas/graphstorm:local