Sign inSign up

diphyx/slurm

By diphyx

•Updated over 1 year ago

Slurm is a job scheduler for high-performance computing clusters

Image
Data science
0

313

diphyx/slurm repository overview

⁠Slurm

Slurm is a job scheduler for high-performance computing clusters. It is designed to manage large clusters of computers and provides a simple interface for users to submit jobs to the cluster.

⁠Template

JWT_KEY:
    label: "JWT Key"
    hint: "More than 32 characters"
    default: "auto"
    required: true
MASTER_HOSTNAME:
    label: "Master Hostname"
    hint: "IP address or hostname"
    default: "localhost"
    required: true
WORKER_HOSTNAMES:
    label: "Worker Hostnames"
    hint: "Comma-separated list"
    default: "localhost"
    required: true
PARTITION_NAMES:
    label: "Partition Names"
    hint: "Comma-separated list"
    default: "main"
    required: true
VOLUME:
    label: "Volume"
    hint: "Starts with /volume"
    default: "/volume"
    required: true
---
name: slurm
services:
    instance:
        image: diphyx/slurm
        privileged: true
        environment:
            - JWT_KEY={{ JWT_KEY }}
            - MASTER_HOSTNAME={{ MASTER_HOSTNAME }}
            - WORKER_HOSTNAMES={{ WORKER_HOSTNAMES }}
            - PARTITION_NAMES={{ PARTITION_NAMES }}
            - DXF_PROXY_MAIN_PORT=6820
        volumes:
            - {{ VOLUME }}:/volume
⁠Guide
  • Run Master and Worker alongside each other
docker run -it --rm --name slurm -p 6817:6817 -p 6818:6818 -p 6819:6819 -p 6820:6820 --privileged diphyx/slurm:latest MASTER,WORKER
  • Run the Master alone
docker run -it --rm --name slurm-master -p 6817:6817 -p 6819:6819 -p 6820:6820 --privileged diphyx/slurm:latest MASTER
  • Run the Worker alone
docker run -it --rm --name slurm-worker -p 6819:6819 --privileged diphyx/slurm:latest WORKER

⁠Sample

  • Create directory
mkdir -p /volume/jobs
  • Create job file
{
    echo '#!/bin/bash'
    echo '#SBATCH --partition=main'
    echo '#SBATCH --job-name=demo'
    echo '#SBATCH --output=output.log'
    echo 'echo "Sleep to 5 seconds"'
    echo 'sleep 5'
    echo 'echo "Done"'
} >> /volume/jobs/demo.sh
  • Submit job
sbatch /volume/jobs/demo.sh
⁠NOTE

This image only works on the DiPhyx⁠ platform.

Tag summary

Content type

Image

Digest

sha256:f0d0f9e41…

Size

168.8 MB

Last updated

over 1 year ago

docker pull diphyx/slurm