ronnyjiang/openfoam_petsc

By ronnyjiang

•Updated over 4 years ago

Openfoam petsc container: include cuda, openmpi, openfoam and petsc.

Image

433

Overview Tags

ronnyjiang/openfoam_petsc repository overview

⁠openfoam PETSc

With our easy-to-run openfoam petsc program.

The container contains the following packages：

OS: Ubuntu 20.04 （LTS）
CUDA: 11.4
OPENMPI: 4.1.2
OPENFOAM: from src --> [https://develop.openfoam.com/Development/openfoam/-/archive/master/openfoam-master.zip⁠]
PETSc: from src --> [https://gitlab.com/petsc/petsc.git⁠]
petsc4Foam: from src --> [https://develop.openfoam.com/modules/external-solver.git⁠]

GPU computing needs to meet the container runtimes：

nvidia-docker
Singularity >= 3.1

⁠x86_64

CUDA driver version >=450.80.02

⁠Support gpu arch

In the CUDA naming scheme, GPUs are named sm_xy, where x denotes the GPU generation number, and y the version in that generation.

Kepler support: sm_35, and sm_37
Maxwell support:sm_50, sm_52 and sm_53
Pascal support: sm_60, sm_61, and sm_62
Volta support: sm_70 and sm_72
Turing support: sm_75
Ampere support: sm_80, sm_86 and sm_87

⁠Usage

Download docker images:

docker push ronnyjiang/openfoam_petsc:latest

Run docker container:

docker run -it --gpus all ronnyjiang/openfoam_petsc

Then run a test case in the container:

cd ThirdParty/external-solver/tutorials/basic/laplacianFoam/flange
./Allrun.init
./Allrun

#Use GPU acceleration

./Allclean
./Allrun.init
sed -i 's/mpiaij/aijcusparse/' run/system/fvSolution-petsc     #modify mat_type
./Allrun

After execution, the directory and log file of the corresponding time point will be generated in the run directory according to the configured calculation time.

eg: log file-> [log.laplacianFoam]

terminal_history

⁠Build docker image

⁠Dockerfile

FROM nvidia/cuda:11.4.0-devel-ubuntu20.04
MAINTAINER YangQi  && RonnyJiang 
LABEL img.version="1.0.1-Alpha" img.version.internal="1.0.2-Alpha"
LABEL Description="Openfoam container, include cuda, openmpi, openfoam and petsc."

ENV DEBIAN_FRONTEND=noninteractive
ENV OMPI_VERSION=4.1.2
ENV OMPI_URL="https://download.open-mpi.org/release/open-mpi/v4.1/openmpi-$OMPI_VERSION.tar.bz2"
ENV OMPI_DIR=/opt/ompi
ENV PATH=$OMPI_DIR/bin:$PATH
ENV LD_LIBRARY_PATH=$OMPI_DIR/lib:$LD_LIBRARY_PATH



COPY ./sources.list /etc/apt/sources.list 
# Install tools required for project
RUN apt-get update && apt-get install -y \
  libblas-dev \
  liblapack-dev \
  m4 \
  flex \
  zlib1g-dev \
  python3 \
  openssh-server \
  git \
  unzip \
  && rm -rf /var/lib/apt/lists/*


# Modify timezone
RUN ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && dpkg-reconfigure -f noninteractive tzdata

# Download and extract package. then compile and install openmpi.
RUN mkdir -p /tmp/ompi && cd /tmp/ompi \ 
  && wget -O openmpi-$OMPI_VERSION.tar.bz2 $OMPI_URL \
  && tar -xjf openmpi-$OMPI_VERSION.tar.bz2 \
  && cd /tmp/ompi/openmpi-$OMPI_VERSION \
  && ./configure --prefix=$OMPI_DIR --with-cuda \
  && make -j8 install


# Shell use bash
SHELL ["/bin/bash", "-c"] 

# Download source code ,make openfoam and inject environment variables.
RUN export OPENFOAM_URL="https://develop.openfoam.com/Development/openfoam/-/archive/master/openfoam-master.zip" \
  && cd /tmp && wget $OPENFOAM_URL \
  && unzip -q openfoam-master.zip -d /opt \
  && mv /opt/openfoam-master /opt/openfoam \
  && cd /opt/openfoam \
  && source etc/bashrc \
  && ./Allwmake -j 8 \
  && echo "source /opt/openfoam/etc/bashrc" >> /etc/bash.bashrc

#RUN cd /opt && git clone https://develop.openfoam.com/Development/openfoam.git \
#  && cd openfoam \
#  && source etc/bashrc \
#  && ./Allwmake -j 8 \
#  && echo "source /opt/openfoam/etc/bashrc" >> /etc/bash.bashrc

# Download source code, make install petsc.
RUN source /opt/openfoam/etc/bashrc \
  && export PETSC_PATH=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER$WM_PRECISION_OPTION$WM_LABEL_OPTION/petsc-git \
  && cd /tmp \
  && git clone https://gitlab.com/petsc/petsc.git \
  && mkdir -p $PETSC_PATH \
  && cd petsc \
  && sed -i "s/genArches\ =\ \[.*/genArches\ =\ ['35',\ '37',\ '50',\ '52',\ '53',\ '60',\ '61',\ '62',\ '70',\ '72',\ '75',\ '80',\ '86',\ '87']/" \
  config/BuildSystem/config/packages/cuda.py \
  && sed -i 's/d+)/w+)/' config/BuildSystem/config/packages/cuda.py \
  && ./configure \
       --with-cuda \
       --with-cuda-gencodearch=all \
       --with-64-bit-indices=0 \
       --with-precision=double \
       --with-debugging=0 \
       --COPTFLAGS=-O3 \
       --CXXOPTFLAGS=-O3 \
       --FOPTFLAGS=-O3 \
       --prefix=$PETSC_PATH \
       PETSC_ARCH=$WM_OPTIONS \
  && make all && make install \
  && echo "export LD_LIBRARY_PATH=\$WM_THIRD_PARTY_DIR/platforms/\$WM_ARCH\$WM_COMPILER\$WM_PRECISION_OPTION\$WM_LABEL_OPTION/petsc-git/lib:\$LD_LIBRARY_PATH" >> /etc/bash.bashrc

# Install petsc4Foam
RUN source /opt/openfoam/etc/bashrc \
  && export PETSC_DIR=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER$WM_PRECISION_OPTION$WM_LABEL_OPTION/petsc-git \
  && export PETSC_LIB=$PETSC_DIR/lib \
  && export LD_LIBRARY_PATH=$PETSC_LIB:$LD_LIBRARY_PATH \
  && cd /opt/openfoam/ThirdParty \
  && git clone https://develop.openfoam.com/modules/external-solver.git \
  && cd /opt/openfoam/ThirdParty/external-solver \
  && ./Allwmake -prefix=openfoam \
  && rm -rf /tmp/*
  
ENV PETSC_OPTIONS="-log_view"
WORKDIR /opt/openfoam

CMD ["/bin/bash"]

⁠build image

step1: generator apt sources.list

tee ./sources.list <<-EOF
# ubuntu 20.04
#aliyun
deb http://mirrors.aliyun.com/ubuntu/ focal main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal-backports main restricted universe multiverse
EOF

step2: build image

docker build -t ronnyjiang/openfoam_petsc .

Tag summary

Recent tags

Content type

Image

Digest

Size

3.8 GB

Last updated

over 4 years ago

docker pull ronnyjiang/openfoam_petsc