wishartlab/phastest-docker repository overview

⁠PHASTEST Docker Cluster

This is a multi-container PHASTEST cluster. The compose file creates named volumes for persistent storage of MySQL data files as well as Slurm state and log directories. multi-container PHASTEST will process queries more rapidly than the single-container option, but is more resource intensive.

To use, please download and unzip Docker Image from https://phastest.ca/databases⁠. Afterward, download slurm-docker-cluster-DB.zip file and place all its contents into /slurm-docker-cluster/phastest-app-docker/DB/ directory.

⁠Containers and Volumes

The compose file will run the following containers:

• mysql
• slurmdbd
• phastest (slurmctld)
• c1-8 (slurmd)

The compose file will create the following named volumes:

• etc_munge ( -> /etc/munge )
• etc_slurm ( -> /etc/slurm )
• slurm_jobdir ( -> /data )
• var_lib_mysql ( -> /var/lib/mysql )
• var_log_slurm ( -> /var/log/slurm )

⁠Building the Docker Image

Image wishartlab/phastest-docker can be pulled from Docker Desktop app, or via command below:

docker pull wishartlab/phastest-docker

Alternatively, from the directory where Dockerfile is present, Docker image may be built locally via running the command below:

docker build -t wishartlab/phastest-docker .

Building the docker image may take up to 20-30 minutes, and up to an hour on M1 Macbook. As such, using image available from the Docker Hub is recommended.

⁠Starting the Cluster

Once the image has been built, use docker compose run command to start the container and immediately begin processing queries.

docker compose run phastest -i {genbank|fasta|contig} -a {accession number} -m {annotation mode} -s {sequence file name} [OPTIONS]...

After a confirmation message, phastest container will start running. Output will be deposited in /phastest-app-docker/JOBS/{job_id} directory.

-i {genbank|fasta|contig}
    Specify input format. Accepted values are 'fasta', 'genbank', or 'contig'.
    If 'genbank' is selected, -a flag and accession number must be provided.
    If 'fasta' or 'contig' is selected, -s flag and the sequence filename must be provided.

-a {accession number}
    Specify accession number for the job. For example, NC_000907.1, KF030445.1, LZPG00000000.1.
    For the WGS sequence, accession number for the master record should be provided.

-m {lite|deep}
    Specify annotation mode. Accepted values are 'deep' or 'lite'.
    If this flag is not specified, it will default to 'lite'.
    'deep' uses Prophage Database and PHAST-BSD Bacterial Database.
    'lite' uses Prophage Database and Swissprot.
    (Note: 'deep' mode may take significantly longer to complete.)

-s {sequence filename}
    Path to the raw FASTA sequence - parsed only if input-type was set to 'fasta' or 'contig'.
    Sequence with the given filename must be present in `/phastest_inputs` folder.
    For example, to run the `test.fna` job, then `test.fna` must be deposited into `/phastest_inputs` and
    docker command should be run like `docker compose run phastest -i fasta -s test.fna ...`.

OPTIONS:
--yes:
    Skips the confirmation message before running the PHASTEST job.

--silent:
    Mutes the output from the PHASTEST terminal.

--phage-only:
    Annotate the predicted phage region only. In default, whole genomes are scanned and annotated.

--single-diamond:
    Only run single diamond blastp for non-phage protein alignment, rather than multiple processes running in parallel.
    Recommended for systems where processing power is limited.

⁠Running Jobs on FASTA/Contig Sequences

For running jobs against FASTA/contig sequences, target sequence must be deposited and present within /phastest-docker/phastest_inputs directory.

When running PHASTEST job, please make sure input type is set as fasta or contig, and full filename (along with the .fna or .fasta extension) is provided with the command.

docker compose run phastest -i fasta -s seq_test.fna

docker compose run phastest -i contig -s contig_test.fna

⁠Running Jobs on GenBank Records

If input type is set as genbank and accession number is provided, then PHASTEST will automatically download record with given accession number and scan for prophage regions.

docker compose run phastest -i genbank -a KF030445.1

docker compose run phastest -i genbank -a LZPG00000000.1

⁠Retrieving Complete Jobs

All complete jobs will be deposited in /phastest-docker/phastest-app-docker/JOBS/{job_id} directory. job_id will be sequence file's basename (in case of FASTA or contig inputs), or accession number (in case of genbank inputs).

⁠Stopping and Restarting the Cluster

You may execute docker compose stop command to stop all containers. Stopped containers will restart during subsequence docker compose run command.

While PHASTEST job is running, you may send SIGINT (Ctrl+C) at any time to terminate current job. All of the scheduled slurm jobs will be terminated also, and phastest container will exit automatically.

⁠Deleting the Cluster

Once the PHASTEST process terminates, stopped container will need to be removed manually. To remove all containers and volumes, run:

docker compose stop
docker container prune

OR

docker compose down

To have PHASTEST process automatically stop all container once the job terminates, replace line 64 through 78 of the docker-entrypoint.sh file with following snippet and rebuild the image using docker build -t phastest-docker . command.

    # Once the container starts up, child nodes will keep running until user pulls down the container itself.
    # Quicker and convenient, but may cause scheduling issues when phastest container is terminated via SIGINT (Ctrl+C) before slurm jobs can finish.
    # exec /usr/sbin/slurmd -Dvvv

    # Once the main phastest container stops, all the child nodes will stop as well.
    # Prevents slurm scheduling issue when slurmctld cuts out while child node is still running.
    exec /usr/sbin/slurmd -Dvvv & 

    while true
    do
        if [[ ! $(scontrol ping | grep "UP") ]]; then
            echo "---> Slurm Node Daemon (slurmd) is no longer active.  Exiting ..."
            exit 0
        fi
    done

In case PHASTEST container terminate via Ctrl+C, deleting the cluster with docker compose down then recreating containers via docker compose run is highly recommended.

⁠Deleting the Cluster

To remove all containers and volumes, run:

docker-compose stop
docker-compose rm -f
docker container prune