kimjoochan/gromacs repository overview

⁠HOW TO USE

You may generate the container from this image and run it as following:

$ nvidia-docker run -v "/**HOME_DIRECTORY**:/home" -v "/etc/localtime:/etc/localtime:ro" -e TZ=**YOUR_TIMEZONE** -v "/tmp/.X11-unix:/tmp/.X11-unix" -e QT_X11_NO_MITSHM=1 --env="DISPLAY" --net=host --gpus all --privileged -itd --name **CONTAINER_NAME** kimjoochan/gromacs:2022.2 /bin/bash
$ docker exec -it **CONTAINER_NAME** /bin/bash

Then, run GROMACS with gmx_mpi prefix.

If gmx_mpi view module gives an error message saying "Can't connect to X server ~~", please try

$ xhost +local:docker

on a terminal outside the container.

Enjoy Simulation!

⁠BUILD INFO

GROMACS version: 2023
Precision: mixed
Memory model: 64 bit
MPI library: MPI (GPU-aware: CUDA)
OpenMP support: enabled (GMX_OPENMP_MAX_THREADS = 128)
GPU support: CUDA
NB cluster size: 8
SIMD instructions: AVX2_256
CPU FFT library: fftw-3.3.8-sse2-avx-avx2-avx2_128
GPU FFT library: cuFFT
Multi-GPU FFT: none
RDTSCP usage: enabled
TNG support: enabled
Hwloc support: hwloc-2.1.0
Tracing support: disabled
C compiler: /usr/bin/cc GNU 9.4.0
C compiler flags: -fexcess-precision=fast -funroll-all-loops -mavx2 -mfma -Wno-missing-field-initializers -pthread -O3 -DNDEBUG
C++ compiler: /usr/bin/c++ GNU 9.4.0
C++ compiler flags: -fexcess-precision=fast -funroll-all-loops -mavx2 -mfma -Wno-missing-field-initializers -pthread -Wno-cast-function-type-strict -fopenmp -O3 -DNDEBUG
BLAS library:
LAPACK library:
CUDA compiler: /usr/local/cuda/bin/nvcc nvcc: NVIDIA (R) Cuda compiler driver;Copyright (c) 2005-2021 NVIDIA Corporation;Built on Mon_May__3_19:15:13_PDT_2021;Cuda compilation tools, release 11.3, V11.3.109;Build cuda_11.3.r11.3/compiler.29920130_0
CUDA compiler flags:-std=c++17;--generate-code=arch=compute_35,code=sm_35;--generate-code=arch=compute_37,code=sm_37;--generate-code=arch=compute_50,code=sm_50;--generate-code=arch=compute_52,code=sm_52;--generate-code=arch=compute_60,code=sm_60;--generate-code=arch=compute_61,code=sm_61;--generate-code=arch=compute_70,code=sm_70;--generate-code=arch=compute_75,code=sm_75;--generate-code=arch=compute_80,code=sm_80;--generate-code=arch=compute_86,code=sm_86;-Wno-deprecated-gpu-targets;--generate-code=arch=compute_53,code=sm_53;--generate-code=arch=compute_80,code=sm_80;-use_fast_math;-Xptxas;-warn-double-usage;-Xptxas;-Werror;-D_FORCE_INLINES;-fexcess-precision=fast -funroll-all-loops -mavx2 -mfma -Wno-missing-field-initializers -pthread -Wno-cast-function-type-strict -fopenmp -O3 -DNDEBUG
CUDA driver: 11.30
CUDA runtime: 11.30

⁠HOW I BUILT THIS IMAGE

Built from nvidia/cuda:11.3.1-devel-ubuntu20.04 via following:

$ nvidia-docker run -v "/**HOME_DIRECTORY**:/home" -v "/etc/localtime:/etc/localtime:ro" -e TZ=Asia/Seoul -v "/tmp/.X11-unix:/tmp/.X11-unix" -e QT_X11_NO_MITSHM=1 --env="DISPLAY" --net=host --gpus all --shm-size=180G --privileged -itd --name GROMACS nvidia/cuda:11.3.1-devel-ubuntu20.04 /bin/bash
$ docker exec -it GROMACS /bin/bash

: apt-get update
: apt-get upgrade
: apt-get install tzdata sudo vim openssh-client libopenblas64-openmp-dev libxinerama-dev python3-dev x11-utils libhwloc-dev gfortran wget git

Then openmpi-4.1.1 tar ball was downloaded and moved to /mnt/ for CUDA-aware MPI support

: wget https://download.open-mpi.org/release/open-mpi/v4.1/openmpi-4.1.1.tar.gz
: tar -xvf openmpi-4.1.1.tar.gz
: cd openmpi-4.1.1
: ./configure --prefix=/usr/local --with-cuda CC=gcc CXX=g++ FC=gfortran
: make all install

(This takes a while!)

Since GROMACS 2023 source code can only build upon CMake versions >= 3.18.1, the latest CMake version was manually installed.
Prior to the installation of CMake 3.26.2, OpenSSL 1.1.1 was installed.

: git clone https://github.com/openssl/openssl.git
: cd openssl-1.1.1d
: ./config
: make
: make test
: make install

Next, CMake 3.26.2 was installed.

: wget https://github.com/Kitware/CMake/releases/download/v3.26.2/cmake-3.26.2.tar.gz
: tar -xvf cmake-3.26.2.tar.gz
: cd cmake-3.26.2
: ./bootstrap
: make
: make install
: hash -r

Next, an user account 'gromacs' was generated & added to sudoers.

: useradd gromacs
: passwd -d gromacs
: usermod -s /bin/bash gromacs
: usermod -aG sudo gromacs
: su gromacs

Then the GROMACS 2023 tar ball was downloaded and moved to /opt

: wget https://ftp.gromacs.org/gromacs/gromacs-2023.tar.gz
: cp gromacs-2023.tar.gz /opt/
: tar -xvf gromacs-2023.tar.gz
: cd gromacs-2023
: mkdir build
: cd build
: cmake .. -DGMX_BUILD_OWN_FFTW=on -DREGRESSIONTEST_DOWNLOAD=on -DGMX_MPI=on -DGMX_GPU=CUDA -DGMX_USE_RDTSCP=on -DGMX_HWLOC=on -DGMX_SIMD=AVX2_256 -DGMX_X11=on 
: make -j 32
: make check
: sudo make install

Finally, the following lines were added to the end of the /etc/bash.bashrc file.

source /usr/local/gromacs/bin/GMXRC
export GMX_GPU_DD_COMMS=true
export GMX_GPU_PME_PP_COMMS=true
export GMX_FORCE_UPDATE_DEFAULT_GPU=true
export GMX_ENABLE_DIRECT_GPU_COMM=true