urgi/docker_caulifinder repository overview

⁠DESCRIPTION OF THE CAULIFINDER DOCKER IMAGE

⁠Dockerfile

The Dockerfile contains the instruction to create the necessary environment for Caulifinder from event_caulifinder master (https://forgemia.inra.fr/urgi-anagen/event_caulifinder⁠). It will also install REPET with its dependencies. This docker builds on the giovtorres/docker-centos7-slurm:17.02.9 image.

⁠Installed tools

• genometools 1.5.8
• BEDtools v2.26.0
• Emboss 6.6.0
• Bioperl
• ncbi blast+ 2.11.0
• ncbi blast 2.2.26
• uclustq1.2.22
• Gblocks 0.91b
• squid-1.9g
• Phyml 3.1
• Phylogenomic master
• fastx_toolkit-0.0.14
• muscle 3.8.31
• trimAl v1.4.rev22
• RepeatMasker 4-0-9-p2
• S-MART 1.1.5
• REPET v2.5.0
  • RMblast
  • piler-2017_03_08
  • RECON 1.08
  • trf404

⁠HOW TO USE THE DOCKER IMAGE ?

⁠Install Docker

/!\ Caulifinder image will not work on debian

• For mac : (macOS 10.12 and higher) https://docs.docker.com/docker-for-mac/install/⁠
• For linux : https://docs.docker.com/install/⁠

⁠Build the docker image

To pull the image from the docker repository, run:

 >docker pull urgi/docker_caulifinder

⁠Prepare your project directory

Create 'myproject' directory, copy in it your genome 'mygenome.fa'

 >mkdir myproject
 >chmod oug+w myproject
 >cd myproject
 >cp PATHTO/mygenome.fa .

⁠Run the container

From 'myproject' folder, execute an interactive bash shell on the container with mount volume on 'myproject' folder.

docker run -ti -h ernie --volume $PWD:/home/centos --name caulifinder_docker urgi/docker_caulifinder:latest

In the container, your are in /home/centos directory containing different databanks and pre-configured files used by caulifinder.

The directory also contains the script to retrieve the CDD database, run:

/home/centos> . forCDD_database.sh

⁠Run Caulifinder

To launch caulifinder Branch A, run:

/home/centos> Caulifinder_branch_A.py -g mygenome.fa -l Caulimoviridae_ref_genomes.fa -b baits.fa -c baits.fa -db Cdd_LE_db/Cdd 1>& Caulifinder_branch_A.txt &

To launch caulifinder Branch B, run:

/home/centos> Caulifinder_branch_B.sh mygenome.fa 1>& Caulifinder_branch_B.txt &

⁠And then?

From the 'caulifinder_docker' container, you quit and still run it with 'ctrl+p ctrl +q' command. So you are returned in your 'myproject' folder with all results (files and folders) :

/home/centos> ctrl+p ctrl+q

From the 'caulifinder_docker' container, you quit and stop it with 'exit' command. So you are returned in your 'myproject' folder with all results (files and folders) :

/home/centos> exit

To come back to 'caulifinder_docker' container, from your 'myproject' folder:

docker container rm -f caulifinder_docker
docker run -ti -h ernie --volume $PWD:/home/centos --name caulifinder_docker urgi/docker_caulifinder:latest

⁠CAULIFINDER README

Caulifinder is a tool that allows discovering Caulimoviridae sequences in plant genomes. It is composed of two branches with complementary purposes: Branch A builds consensus sequences of endogenous Caulimoviridae Branch B produces a phylogenetic analysis of marker genes.

Please read the following for more details regarding the process and usage of each branch.

⁠BRANCH A

Branch A - Caulifinder_branch_A.py - aims at constructing a library of consensus sequences that are representative of repetitive elements with significant homology to Caulimoviridae found in a plant genome. This library typically contains complete Caulimoviridae genomes, almost complete genomes and genomic fragments. Branch A also builds groups of consensus sequences sharing high similarity in order to establish tentative biological links between consensus. This shall allow for instance grouping the different parts of multipartite genomes and connecting short fragments with longer sequences.

The main steps of branch A are as follows:

• step 1 : Build a subgenome enriched in Caulimoviridae-related loci
• step 2 : Launch TEdenovo to generate consensus sequences that are representative of repetitive elements in the subgenome.
• step 3 : Analysis of the main features and clustering of Caulimoviridae sequences
• step 4 : Genome annotation with RepeatMasker

The following files are provided to run branch A:

• "Caulimoviridae_ref_genomes.fa" contains reference Caulimoviridae genome sequences (nt)
• "baits.fa" contains translated ORFs from Caulimoviridae genomes as well as reverse transcriptase (RT) and ribonuclease H1 (RH) sequences from Copia and Gypsy elements obtained from the Gypsy database. It is used to identify and discard the sequences that have best BLAST hit against Copia or Gypsy sequences.
• "TEdenovo.cfg". The running directory should also contain the configuration file for TEdenovo (launched at step 2) named "TEdenovo.cfg". This file should be modified to specify a working directory if different from default (project_dir:/home/centos on line #12). Note that several TEdenovo parameters can be modified in TEdenovo.cfg (see the TEdenovo tutorial at https://urgi.versailles.inra.fr/Tools/REPET/TEdenovo-tuto⁠).
• The CDD database in the Cdd_LE_db/ directory and the table "cddid_all.tbl" allowing the conversion of Cdd identifiers to proper domain names

Before launching: The working directory must also contain the input genome provided by user (headers should NOT contain space characters or symbols such as "=", ";", ":", "|"...).

Mandatory parameters:

• -g: input_genome (fasta format)
• -db: path to CDD library (e.g. user/db/Cdd)
• -l: reference Caulimoviridae genomes in fasta format
• -b: library of amino acid sequences used to filter out false positives, in fasta format
• -c: library of amino acid sequences used to classify consensus sequences, in fasta format

Additional options

• -r: name and path of CDD description table (e.g. user/db/Cdd/cddid_all.tbl) (default is using cddid_all.tbl present in project directory)
• -S: identity threshold used by BLASTclust (default = 90, meaning 90% identity)
• -L: coverage threshold used by BLASTclust (default = 0.9, meaning 90% coverage)
• -e: evalue for initial tBLASTx (default = 10-3)
• -X: extension value, length of upstream and downstream hit extension, in bp (default is set to 2000 bp)
• -ch: filter consensus with potential TE domains (gag, integrase, transposase ...)
• -hsp: minimum number of high-scoring pairs in a group to build a consensus, either 5 (default True) else 3 (False)
• -bl: Add to final selection all the sequences from the blastclust clusters which contain at least one consensus classified as Caulimoviridae (default False). Be careful, if True, this option may lead to the selection of false positives, for exploratory use only.

To launch, in background, Caulifinder_branch_A.py on myGenome.fa, using default names for outputs, collect the standard output/standard error in Caulifinder_branch_A.txt, run:

Caulifinder_branch_A.py -g mygenome.fa -l Caulimoviridae_ref_genomes.fa -b baits.fa -c baits.fa -db Cdd_LE_db/Cdd 1>& Caulifinder_branch_A.txt &

To launch, in background, Caulifinder_branch_A.py using -o option and an extension set to 3000 bp, run:

Caulifinder_branch_A.py -g mygenome.fa -l Caulimoviridae_ref_genomes.fa -b baits.fa -c baits.fa -db Cdd_LE_db/Cdd -X 3000 -o results_3000.csv 1>& Caulifinder_branch_A_3000.txt &

Main output files: in summary_branchA/

• Caulimoviridae.rps - output of the conserved protein domains detection using the rpstblastn tool against the CDD database
• Caulimoviridae_BestHits.fa in case of bl option - consensus sequences with best hit against Caulimoviridae used to identify Caulimoviridae clusters
• Caulimoviridae_all.fa - all consensus sequences belonging to a Caulimoviridae cluster
• FilteredChimeras.txt - consensus which have been discarded because they were suspected of being chimeras
• summary_table.csv - table summarizing the main features of each consensus sequences belonging to a Caulimoviridae cluster
• RepeatMasker output folder including ".tbl" file indicating genome coverage and ".gff" file indicating the position of RM annotation.

⁠BRANCH B

Branch B - Caulifinder_branch_B.sh - aims at collecting amino acid RT sequences from endogenous Caulimoviridae and to use these, in combination to reference sequences, to produce a phylogenetic tree. The classification suggested by the pylogenetic tree can be more accurate than that proposed by best hit in Branch A, especially with sequences that do not fall within any reference clade. In addition, branch B also allows detecting low copy RT loci by contrast to branch A which inherently only collects repetitive elements.

The main steps of branch B are as follows:

• step 1 : Mine input genome for potential Caulimoviridae RT sequences
• step 2 : Cluster RT sequences and select representatives
• step 3 : Build a phylogenetic tree

The following files are provided to run branch B:

• "Caulimo_RT_probes.fa" is composed of a set of aa RT sequences taken from the known breadth of Caulimoviridae diversity (one RT per genus or proposed genera)
• "Tree_RT_set.fa" comprises aa RT sequences taken from an MSA used to build a reference phylogenetic tree of Caulimoviridae with outgroup sequences from retroviruses and LTR retrotransposons.

In addition, the input genome file in fasta format should be in the running directory (headers in genome should NOT contain space characters or symbols such as "=", ";", ":", "|"...).

To launch branch B, run:

Caulifinder_branch_B.sh mygenome.fa 1>& launch.txt &

Main output files: in summary_branchB/

• RT_clusters_all.fa: RT library clustered with UCLUST
• Representative_RT.fa: representative sequences from each cluster
• candidate_Caulimoviridae_RT.fa: selected representative sequences
• final_MSA.fa: MSA obtained at the last MUSCLE+TrimAL iteration and filtered by Gblocks. This MSA is used to build the phylogenetic tree.
• final_tree.txt: Phylogenetic tree (Newick) proposed to help classifying RT sequences from endogenous Caulimoviridae found in input genome.

All intermediate files created during the workflow are kept. For instance, cluster sizes can be checked in UCLUST output file "results.clstr"