Sign inSign up

skchronicles/metavirs

By skchronicles

•Updated over 1 year ago

Viral Metagenomics Assembly and Annotation Pipeline

Image
1

1.6K

skchronicles/metavirs repository overview

⁠Overview

Welcome to metavirs! Before getting started, we highly recommend reading through metavirs' documentation⁠.

The ./metavirs pipeline is composed of several interrelated sub-commands to set up and run the pipeline across different systems. Each of the available sub-commands performs different functions:

metavirs is a comprehensive viral metagenomics pipeline to assemble, annotate, and classify environmental samples. It relies on technologies like Singularity to maintain the highest level of reproducibility. The pipeline consists of a series of data processing and quality-control steps orchestrated by Snakemake, a flexible and scalable workflow management system, to submit jobs to a cluster.

The pipeline is compatible with data generated from Illumina short-read sequencing technologies. As input, it accepts a set of FastQ files and can be run locally on a compute instance, on-premise using a cluster, or on the cloud (feature coming soon!). A user can define the method or mode of execution. The pipeline can submit jobs to a cluster using a job scheduler like SLURM, or run on AWS using Tibanna (feature coming soon!). A hybrid approach ensures that the pipeline is accessible to all users.

Before getting started, we highly recommend reading through the usage section of each available sub-command.

For more information about issues or troubleshooting a problem, please check out our FAQ prior to opening an issue on Github.

⁠Dockerfile

Here is the Dockerfile of the latest version of metavirs:

# Base image for metavirs (v0.2.0),
# uses Ubuntu Focal (LTS)
FROM ubuntu:20.04

# Depedencies of metvirs:
#  - CAT/5.2.3            # from src, installed: 5.2.3
#  - cutadapt/1.18        # apt-get, default: 2.8
#  - bowtie/2-2.3.4.1     # apt-get, deafult: 2.3.5.1
#  - fastq_screen/0.14.1  # from src, installed: 0.15.2
#  - fastqc/0.11.5        # apt-get, deafault: 0.11.9
#  - kraken/2.1.1         # from src, installed: 2.1.2
#  - kronatools/2.8       # from src, installed: 2.8.1
#  - megahit/1.2.9        # from src, installed: 1.2.9
#  - quast/5.0.2          # from src, installed: 5.1.0rc1 
#  - multiqc/1.12         # from src, installed: 1.12
#  - python/3.8           # apt-get, default: 3.8.2
#  - samtools/1.15        # apt-get, default: 1.10
#  - seqtk/1.3            # apt-get, deafult: 1.3
#  - spades/3.15.4        # from src, installed: 3.15.4
#  - ucsc/418             # personal mirror: snapshot Nov 2nd, 2020
#  - blastn/2.13.0+       # download binary, installed: 2.13.0+
#  - fastx_toolkit/0.0.13 # download binary, installed: 0.0.13
LABEL [email protected]

############### INIT ################
# Create Container filesystem specific 
# working directory and opt directories
# to avoid collisions with the host's
# filesystem, i.e. /opt and /data
RUN mkdir -p /opt2 && mkdir -p /data2
WORKDIR /opt2 

# Set time zone to US east coast 
ENV TZ=America/New_York
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime \
        && echo $TZ > /etc/timezone

############### SETUP ################
# This section installs system packages 
# required for your project. If you need 
# extra system packages add them here.
RUN apt-get update \
    && apt-get -y upgrade \
    && DEBIAN_FRONTEND=noninteractive apt-get install -y \
        bc \
        build-essential \
        bzip2 \
        cmake \
        cpanminus \
        curl \
        gawk \
        git \
        g++ \
        gcc \
        gfortran \
        gzip \
        locales \
        make \
        openjdk-8-jdk \
        pandoc \
        # perl/5.30.0
        perl \
        libgd-gd2-perl \
        parallel \
        pigz \
        # python/2.7.17
        python2 \
        # python/3.8.2
        python3 \
        python3-pip \
        samtools \
        unzip \
        wget \
        zlib1g-dev \
        # metaquast dependencies below
        pkg-config \
        libfreetype6-dev \
        libpng-dev \
        libboost-all-dev \
        # spades dependencies below
        bamtools \
        bwa \
        libhat-trie0 \
        libnlopt0 \
        libssw0 \
        libbz2-dev \
        liblzma-dev \
    && apt-get clean && apt-get purge \
    && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Make python3 the default interpreter
# and install Python Packages, metaquast 
# requires matplotlib python package, 
# install multiqc/1.12 (latest), spades
# requires joblib, pyyaml (yaml)
RUN ln -sf /usr/bin/python3.8 /usr/bin/python
RUN pip3 install --upgrade pip \
	&& pip3 install argparse \
	&& pip3 install numpy \
        && pip3 install pysam \
	&& pip3 install scipy \
        && pip3 install matplotlib \
        && pip3 install setuptools \
        && pip3 install multiqc==1.12 \
        && pip3 install pyyaml \
        && pip3 install joblib

# Installl Perl Libraries, GD::Graph::bars 
# is required for fastq_screen, metaquast
# requires Time::HiRes, NOTE: apt-get the 
# package libgd-gd2-perl prior to installing
# GD::Graph::bars library due to a known bug
RUN cpanm FindBin Term::ReadLine \
        && cpanm GD::Graph::bars \
        && cpanm Time::HiRes

# Set the locale
RUN localedef -i en_US -f UTF-8 en_US.UTF-8

############### INSTALL ################
# Install any bioinformatics tools
# available with apt-get on Ubuntu/20.04
RUN apt-get update \
    && apt-get -y upgrade \
    && DEBIAN_FRONTEND=noninteractive apt-get install -y \
        # cutadapt/2.8
        cutadapt \
        # bedtools/2.27.1
        bedtools \
        # bowtie/1.2.3
        bowtie \
        # bowtie2/2.3.5.1
        bowtie2 \
        # fastqc/0.11.9
        fastqc \
        # samtools/1.10
        samtools \
        # seqtk/1.3
        seqtk \
    && apt-get clean && apt-get purge \
    && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

############### MANUAL ################
# Install fastq_screen/0.15.2 manually, 
# not available to apt-get on Ubuntu/20.04.
# Dependencies already satisfied:
# https://github.com/StevenWingett/FastQ-Screen/tree/master/docs#installation
RUN wget https://github.com/StevenWingett/FastQ-Screen/archive/refs/tags/v0.15.2.tar.gz \
        && tar -xvzf /opt2/v0.15.2.tar.gz \
        && rm /opt2/v0.15.2.tar.gz
# Add fastq_screen to $PATH
ENV PATH="/opt2/FastQ-Screen-0.15.2:$PATH"
WORKDIR /opt2

# Install kraken/v2.1.2 (latest) manually,
# not available to apt-get on Ubuntu/20.04.
# Dependencies already satisfied:
# https://github.com/DerrickWood/kraken2/releases 
RUN wget https://github.com/DerrickWood/kraken2/archive/v2.1.2.tar.gz \
	    && tar -xvzf v2.1.2.tar.gz \
	    && rm /opt2/v2.1.2.tar.gz \
	    && cd /opt2/kraken2-2.1.2 \
 	    && ./install_kraken2.sh . \
	    && chmod a+rX /opt2/kraken2-2.1.2/*
# Add kraken2 to PATH
ENV PATH="/opt2/kraken2-2.1.2:$PATH"
WORKDIR /opt2

# Install kronatools/v2.8.1) manually, 
# not available to apt-get on Ubuntu/20.04.
# Dependencies already satisfied:
# https://github.com/marbl/Krona/releases
# Build Krona Taxonomy Databases
# ktImportTaxonomy relies on NCBI taxonomy: 
# https://github.com/marbl/Krona/wiki/Installing 
RUN wget https://github.com/marbl/Krona/releases/download/v2.8.1/KronaTools-2.8.1.tar \
	    && tar -xvf KronaTools-2.8.1.tar  \
	    && rm KronaTools-2.8.1.tar \ 
	    && cd KronaTools-2.8.1/ \
	    && ./install.pl  \
	    && mkdir -p /opt2/KronaTools-2.8.1/taxonomy \
	    && ./updateTaxonomy.sh
WORKDIR /opt2

# Install UCSC tools from personal mirror,
# Snapshot was taken on November 2nd, 2020
# https://github.com/skchronicles/ucsc_mirror/
RUN wget https://github.com/skchronicles/ucsc_mirror/raw/main/faSplit \
        && chmod a+x faSplit

# Install megahit/1.2.9 (latest) manually,
# not available to apt-get on Ubuntu/20.04.
# Dependencies already satisfied:
# https://github.com/voutcn/megahit
RUN wget https://github.com/voutcn/megahit/releases/download/v1.2.9/MEGAHIT-1.2.9-Linux-x86_64-static.tar.gz \
        && tar -xvzf MEGAHIT-1.2.9-Linux-x86_64-static.tar.gz \
        && rm /opt2/MEGAHIT-1.2.9-Linux-x86_64-static.tar.gz
# Add megahit to $PATH
ENV PATH="/opt2/MEGAHIT-1.2.9-Linux-x86_64-static/bin:$PATH"
WORKDIR /opt2

# Install metaquast/5.1.0rc1 manually, 
# not available to apt-get on Ubuntu/20.04.
# Dependencies already satisfied:
# https://github.com/ablab/quast
RUN wget https://github.com/ablab/quast/releases/download/quast_5.1.0rc1/quast-5.1.0rc1.tar.gz \
        && tar -xvzf quast-5.1.0rc1.tar.gz \
        && rm /opt2/quast-5.1.0rc1.tar.gz \
        && cd /opt2/quast-5.1.0rc1/ \
        && ./setup.py install
WORKDIR /opt2

# Install CAT/5.2.3 manually,
# not available to apt-get on Ubuntu/20.04.
# CAT has two dependencies that have 
# not been satisfied:
#  - prodigal
#  - diamond (bundled with CAT Prep ref)
# @Install prodigal/2.6.3
RUN wget https://github.com/hyattpd/Prodigal/releases/download/v2.6.3/prodigal.linux \
        && chmod +x /opt2/prodigal.linux \
        && mv /opt2/prodigal.linux /opt2/prodigal
# @Install CAT/5.2.3
RUN wget https://github.com/dutilh/CAT/archive/refs/tags/v5.2.3.tar.gz \
        && tar -xvzf v5.2.3.tar.gz \
        && rm /opt2/v5.2.3.tar.gz
# Add CAT to $PATH
ENV PATH="/opt2/CAT-5.2.3/CAT_pack:$PATH"
WORKDIR /opt2

# Install spades/3.15.4 manually, latest version
# not available to apt-get on Ubuntu/20.04.
# Dependencies already satisfied:
RUN wget http://cab.spbu.ru/files/release3.15.4/SPAdes-3.15.4-Linux.tar.gz \
        && tar -xvzf SPAdes-3.15.4-Linux.tar.gz  \
        && rm /opt2/SPAdes-3.15.4-Linux.tar.gz 
# Add metaspades to $PATH
ENV PATH="/opt2/SPAdes-3.15.4-Linux/bin:$PATH"
WORKDIR /opt2

# Install blastn/2.13.0+ from NCBI FTP,
# not available to apt-get on Ubuntu/20.04.
# Dependencies already satisfied:
RUN wget https://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ncbi-blast-2.13.0+-x64-linux.tar.gz \
        && tar -zvxf ncbi-blast-2.13.0+-x64-linux.tar.gz ./ncbi-blast-2.13.0+/bin/blastn \
        && rm ncbi-blast-2.13.0+-x64-linux.tar.gz
# Add blastn to $PATH
ENV PATH="/opt2/ncbi-blast-2.13.0+/bin:$PATH"
WORKDIR /opt2

# Install fastx_toolkit/0.0.13 from CSHL,
# not available to apt-get on Ubuntu/20.04.
# Dependencies already satisfied:
RUN mkdir fastx_toolkit \
        && cd fastx_toolkit \
        && wget http://hannonlab.cshl.edu/fastx_toolkit/fastx_toolkit_0.0.13_binaries_Linux_2.6_amd64.tar.bz2 \
        && tar -jvxf fastx_toolkit_0.0.13_binaries_Linux_2.6_amd64.tar.bz2 ./bin/fastq_to_fasta \
        && rm fastx_toolkit_0.0.13_binaries_Linux_2.6_amd64.tar.bz2
# Add fastq_to_fasta to $PATH
ENV PATH="/opt2/fastx_toolkit/bin:$PATH"
WORKDIR /opt2

################ POST #################
# Add Dockerfile and export environment 
# variables and set java8 as default with links
# to alternative versions
ADD Dockerfile /opt2/metavirs.dockerfile
RUN chmod -R a+rX /opt2 \
        && ln -s /usr/lib/jvm/java-8-openjdk-amd64/bin/java /usr/bin/java8 
ENV PATH="/opt2:$PATH"
WORKDIR /data2

⁠Building the Docker Image

Directly below are instructions for building an image using the provided Dockerfile:

# See a listing of images on computer
docker image ls

# Build from Dockerfile
docker build --no-cache -f Dockerfile --tag=metavirs:v0.1.0 .

# Testing, take a peek inside
docker run -ti metavirs:v0.1.0 /bin/bash

# Updating Tag  before pushing to DockerHub
docker tag metavirs:v0.1.0 skchronicles/metavirs:v0.1.0
docker tag metavirs:v0.1.0 skchronicles/metavirs         # latest

# Check out new tag(s)
docker image ls

# Push newly tagged image to DockerHub
docker push skchronicles/metavirs:v0.1.0
docker push skchronicles/metavirs:latest
⁠Other Recommended Steps

Scan your image for known vulnerabilities:

docker scan metavirs:v0.1.0

Please Note: Any references to skchronicles should be replaced your username if you would also like to push the image to a non-org account. Also, a copy of the Dockerfile used to build this image can be found in the container's /opt2 directory.

Tag summary

Content type

Image

Digest

sha256:9a2ae5abd…

Size

1.1 GB

Last updated

over 1 year ago

docker pull skchronicles/metavirs