shl198/sc_ppl repository overview

FROM rocker/tidyverse:4.3

ENV LD_LIBRARY_PATH /usr/local/lib/R/lib/:$LD_LIBRARY_PATH

RUN apt update && apt install -y python3-pip libglpk-dev libhdf5-dev

RUN pip install jupyterlab scanpy muon rpy2 anndata2ri scirpy gprofiler-official harmonypy mvtcr omnipath decoupler scarches gseapy squidpy mofapy2 scanorama bioinfokit mofax

RUN Rscript -e
"if (!require("BiocManager", quietly = TRUE))
{install.packages("BiocManager")};
install.packages('Seurat');
BiocManager::install(c('scran','MAST','monocle','ComplexHeatmap','slingshot','clusterExperiment','DropletUtils','gam','miloR','lme4'))"

CMD ["/bin/bash"]

Use container to run jupyterlab/rstudio in HPC

How to run jupyterlab

1. Login to HPC server using your username and password through noMachine and open a terminal inside it (It’s recommended to use NoMachine because if you quit NoMachine, the running processes would not be interrupted).
2. Choose a place to create a folder that will be used to store your container.
3. Download docker container using singularity in HPC. • singularity pull docker://shl198/sc_ppl:202303. (After this command, a singularity image file named sc_ppl_202303.sif is stored in the current directory. This is a container for single cell pipelines, you can use any public docker container that have jupyterlab installed).
4. Build sandbox using singularity in HPC (this is to make sure you can install new packages in the container in HPC) • singularity build –sandbox sc_ppl sc_ppl_2023.sif. (This step will create a folder named sc_ppl)
5. In noMachine terminal, login to other compute nodes (you can use lsload in terminal to check what nodes are available in HPC, I list two options here, option one is the standard way to run, option two is easier but cheat way to do): • (Standard way to start an interactive bsub) bsub -M 20G -n 1 -q medium -m 'hpccpu200.com' -Is bash (you can check documentation of bsub to request the number of cores and memory you need, -M sets memory, -n sets number of cores, -q set the queue, -m choose the computing node, this way doesn’t apply to all nodes, you need to select the correct nodes). • ssh hpccpu200 (you can change to other nodes based on the load)
6. Start the sandbox • singularity shell -B /:/media --writable sc_ppl. (This step will access the container)
7. Run jupyterlab with out opening browser • jupyter-lab --no-browser --port 8080 • copy the url shown in the terminal starts with: http://localhost:8080/lab?token=⁠
8. In your own laptop open a terminal that allows ssh connection (one example is mobaxterm). • Run command in terminal ssh -L 8080:localhost:8080 [email protected]⁠
9. In the browser of your own laptop, open the url in step 7b.

How to run Rstudio in HPC

1. Login to HPC server using your User ID and password through noMachine and open a terminal inside it. (It’s recommended to use NoMachine because if you quit NoMachine, the running processes would not be interrupted).
2. Choose a place to create a folder that will be used to store your container.
3. Download docker container using singularity in HPC. • singularity pull docker://bioconductor/bioconductor_docker:RELEASE_3_17 (After this command, a singularity image file named bioconductor_docker_RELEASE_3_17.sif is stored in the current directory. This is an image for single cell pipelines, you can use any public docker container that have Rstudio installed, another popular docker container is the tidyverse container).
4. Build sandbox using singularity in HPC (this is to make sure you can install new packages in the container in HPC) • singularity build --sandbox bioconductor bioconductor_docker_RELEASE_3_17.sif (This step will create a folder named bioconductor)
5. In noMachine terminal, login to other compute nodes (you can use lsload in terminal to check what nodes are available in HPC, I list two options here, option one is the standard way to run, option two is easier but cheat way to do): • (Standard way to start an interactive bsub) bsub -M 20G -n 1 -q medium -m 'hpccpu200.com' -Is bash (you can check documentation of bsub to request the number of cores and memory you need, -M sets memory, -n sets number of cores, -q set the queue, -m choose the computing node, this way doesn’t apply to all nodes, you need to select the correct nodes). • ssh hpccpu200 (you can change to other nodes based on the load)
6. Start the sandbox • PASSWORD='bioc' singularity run -w -B /:/media bioconductor rserver --auth-none=0 --auth-pam-helper-path=pam-helper --server-user yourID
7. In your own laptop open a terminal that allows ssh connection (one example is mobaxterm). • Run command in terminal ssh -L 8787:localhost:8787 [email protected]⁠.
8. In the browser of your own laptop use the following url to access rstudio • http://localhost:8787/⁠ • Enter username and password (in this example user is yourID, password is bioc).