bioedge/edge_dev repository overview

⁠EDGE Bioinformatics

This is docker image for development version of EDGE Bioinformatics, a product of collaboration between Los Alamos National Laboratory and the Naval Medical Research Center sponsored by the Defense Threat Reduction Agency.

EDGE is a highly adaptable bioinformatics platform that allows laboratories to quickly analyze and interpret genomic sequence data. The bioinformatics platform allows users to address a wide range of use cases including assay validation and the characterization of novel biological threats, clinical samples, and complex environmental samples.

⁠How to use this image

⁠Install Docker

See Docker at https://www.docker.com/⁠

⁠Obtain the docker image

$ docker pull bioedge/edge_dev

⁠Obtain inital mysql database

$ wget https://edge-dl.lanl.gov/EDGE/dev/VirtualMachine/edge_dev_init_mysql_EDGE_input_for_edge_docker.tgz

⁠Download EDGE database from web server

⁠You will need ~300GB disk space for all databases

  # Pipeline database is ~18 Gb and contains the other databases needed for EDGE
  wget -c https://edge-dl.lanl.gov/EDGE/dev/edge_dev_pipeline_databases.tgz

  # BWA index is ~41Gb and contains the databases for bwa taxonomic identification pipeline
  wget -c https://edge-dl.lanl.gov/EDGE/dev/edge_dev_bwa_index.tgz

  # HOST genomes BWA index is ~48Gb for Host removal, including human, bacteria, phiX, viruses, invertebrate vectors of human pathogens
 wget -c https://edge-dl.lanl.gov/EDGE/dev/edge_dev_HostIndex.tgz

 # NCBI Genomes is ~21Gb and contain the full genomes for prokaryotes and some viruses
 wget -c https://edge-dl.lanl.gov/EDGE/dev/edge_dev_NCBI_genomes.tgz

 # GOTTCHA database is ~16Gb and contains the custom databases for the GOTTCHA taxonomic identification pipeline
 wget -c https://edge-dl.lanl.gov/EDGE/dev/edge_dev_GOTTCHA_db.tgz

 # Amplicon database is ~78Mb and contains the databases for Qiime 16s and 18s ITS pipeline
 wget -c https://edge-dl.lanl.gov/EDGE/dev/edge_dev_amplicons_db.tgz

 # NT database is ~25Gb and contains the NCBI nt database for contig identification
 wget -c https://edge-dl.lanl.gov/EDGE/dev/edge_dev_nt_20160426.tgz

 # ShortBRED database is ~27Mb and contains the databases used by ShortBRED for virulence factors and read based antibiotic resistance analysis
 wget -c https://edge-dl.lanl.gov/EDGE/dev/edge_dev_ShortBRED_Database.tgz

 # Diamond database is ~16Gb and contains the databases from RefSeq for protein based taxonomic identification
 wget -c https://edge-dl.lanl.gov/EDGE/dev/edge_dev_diamond_db.tgz

 # MetaPhlAn2 database is 1.1Gb contains the databases used for the MetaPhlAn2 taxonomic identification pipeline 
 wget -c https://edge-dl.lanl.gov/EDGE/dev/edge_dev_metaphlan2DB.tgz

 # GOTTCHA2 databases are 27Gb and 883Mb and contains the custom databases for the GOTTCHA2 taxonomic identification pipeline
 wget -c https://edge-dl.lanl.gov/EDGE/dev/edge_dev_GOTTCHA2_bac_db.tgz
 wget -c https://edge-dl.lanl.gov/EDGE/dev/edge_dev_GOTTCHA2_virus_db.tgz

 (Optional)
 # Other Host bwa index ~18Gb for host removal, including pig, sheep, cow, monkey, hamster. and goat.
 wget -c https://edge-dl.lanl.gov/EDGE/dev/edge_dev_otherHostIndex.tgz

  # For machine with < 32Gb memory, we suggest to use the smaller BWA index (~14Gb) and contains the databases for bwa taxonomic identification pipeline
  wget -c https://edge-dl.lanl.gov/EDGE/dev/edge_dev_bwa_mini_index.tgz

  # Decompressed each tar.gz file (tar -xzvf) for being used later

⁠Start EDGE bioinformatics instance

    $ docker run -d --privileged=true --security-opt "seccomp:unconfined" \
    --cap-add=SYS_ADMIN --cap-add=SYS_PTRACE  \
    -v /path/to/mysql:/var/lib/mysql \
    -v /path/to/database:/home/edge/database \
    -v /path/to/EDGE_output:/home/edge/EDGE_output \
    -v /path/to/EDGE_input:/home/edge/EDGE_input \
    -v /path/to/EDGE_report:/home/edge/EDGE_report \
    -p 80:80 -p 8080:8080 --name edge bioedge/edge_dev

Wait for few seconds for the docker image to start EDGE service and Open http://localhost/⁠ on the browser to start experience EDGE.

• The -v /path/to/mysql:/var/lib/mysql part of the command mounts the /my/own/mysql (obtain from the link above and chown to mysql:mysql if needed.) directory from the underlying host system as /var/lib/mysql inside the container, where MySQL by default will write its data files. Using this to persist the database data in the host. You can use data volume instead and it described in the "Note" section .
• The -v /path/to/database:/home/edge/database mounts the databse obtained from the above download step.
• The -v /path/to/EDGE_input://home/edge/EDGE_input mounts the EDGE input directory structure (obtain from the git clone above) to persist the input/upload files/user projects in the host.
• The -v /path/to/EDGE_output://home/edge/EDGE_output mounts the EDGE output directory to persist the output files in the host.
• The -p host:container bind the host port 80 and 8080 to container port 80 and 8080 inside the container. You can change the 80 and 8080 to fit your host system requirements.
• In some cases, you need to set the directory into 0777 mode. Please try opening up the directory permission if you run into trouble.

⁠Default credentials

• EDGE user: [email protected]⁠/admin_docker

• For security, you may need update the credentials if the server will be used by others or public.

⁠Note

• The image size is around 13.5GB: If you have issue pulling this image, you may increase the basesize when launch docker daemon or using different Storage Driver⁠. see issue⁠.
• This image is built on top of offical CentOS 7 Base Image (7.3.1611), and is officially supported on Docker version 17.12.0-ce-mac49 (21995) (API version: 1.35).
• The user management can be accessed by http://localhost:8080/userManagement⁠ if host port is 8080
• There is an issue to mount host volume using MAC OSX. https://github.com/boot2docker/boot2docker/issues/581⁠. However, there is a workaround to use data volume container instead.

    * Use data volume container (a.k.a: OS independent.)
    $ docker pull bioedge/edge_mariadb
    $ docker create --name mysql_data --volume /var/lib/mysql  bioedge/edge_mariadb
    $ docker run -d-privileged=true --security-opt "seccomp:unconfined" --cap-add=SYS_ADMIN --cap-add=SYS_PTRACE --volumes-from mysql_data -v /path/to/database:/home/edge/database -v /path/to/EDGE_output:/home/edge/EDGE_output -v /path/to/EDGE_input:/home/edge/EDGE_input/public -p 80:80 -p 8080:8080 --name edge bioedge/edge_dev

⁠Commands for checking status and error log

• Check the mariadb status in container: $ docker exec edge systemctl status mariadb.service

where "edge" is the container name when user docker run it with --name flag

• And user management system service status: $ docker exec edge systemctl status tomcat.service

• For the Apache web server status and log: $ docker exec edge systemctl status httpd.service $ docker exec edge tail /var/log/httpd/error_log $ docker exec edge tail /var/log/httpd/access_log

⁠Customize the docker0 bridge

Docker is hard coded to look for 172.17.0.1. If the ip address conflict with the subnet of your wifi, you may edit the /etc/docker/daemon.json as describe here⁠

⁠Contact Info

Chien-Chi Lo: [email protected]⁠ Paul Li: [email protected]⁠

⁠Citation

Po-E Li, Chien-Chi Lo, Joseph J. Anderson, Karen W. Davenport, Kimberly A. Bishop-Lilly, Yan Xu, Sanaa Ahmed, Shihai Feng, Vishwesh P. Mokashi, Patrick S.G. Chain; Enabling the democratization of the genomics revolution with a fully integrated web-based bioinformatics platform, Nucleic Acids Research, Volume 45, Issue 1, 9 January 2017, Pages 67–80, https://doi.org/10.1093/nar/gkw1027⁠