knowengdev/samples_clustering_pipeline repository overview

⁠KnowEnG's Samples Clustering Pipeline

This is the Knowledge Engine for Genomics (KnowEnG), an NIH BD2K Center of Excellence, Samples Clustering Pipeline.

This pipeline clusters the columns of a given spreadsheet, where spreadsheet's columns correspond to sample-labels and rows correspond to gene-labels.

There are four clustering methods that one can choose from:

Note: all of the clustering methods mentioned above use the non-negative matrix factorization (nmf) as the main clustering algorithm.

If a pheotype data file is included this pipeline evaluates the clustering result.

There are two evaluation methods:

---

⁠How to run this pipeline's docker image with Our data

---

1. Install Docker - follow the instructions in this link. ============

Install Docker Engine⁠

1. Get the current Docker Image. On the command line with internet connection: ============

docker pull knowengdev/samples_clustering_pipeline: 04_09_2018

1. Create or change to a directory to hold the output data. ============

mkdir local_results and / or cd local_results

1. Start the docker image connected to the (local_results) directory. =========== docker run -v local_results:/home/test/run_dir -it knowengdev/samples_clustering_pipeline: 04_09_2018

2. At the docker image command prompt change to the test directory. =========== .../home# cd test

3. Set up the environment. ============= .../home/test# make env_setup

---

1. Use one of the following "make" commands to select and run a clustering option: ===========

---

⁠How to run this pipeline with Your data.

---

*** Perform steps 1-3 as described above and use the local_results directory ***

⁠Create a custom YAML file, move that and your spreadsheet to the local_results directory

*** see the git hub directions *** Samples Clustering ReadMe on github⁠

*** make sure file named by YAML key "spreadsheet_name:" is in your local_results directory *** *** the path-names inside docker depend on the way you "mount" the run directory in step 4 *** *** therefore you may have to change the YAML path's to ../../ instead of ../ *** *** make sure your custom YAML file is in the local_results directory ***

1. Start docker with the container connected to your (local_results) directory. =========== docker run -v local_results:/home/test/run_dir -it knowengdev/samples_clustering_pipeline:04_09_2018

2. Change to the directory mounted in step 4. =========== .../home# cd ./test/run_dir .../home/test/run_dir# ls

*** the files in the local_results directory will be visible in this directory ***

1. Run Samples Clustering with the options in your custom YAML file. ========= python3 ../../src/samples_clustering.py -run_directory ./ -run_file your_custom_run_file.yml

---

⁠Description of "run_parameters" file

---

gg_network_name = STRING_experimental_gene_gene.edge spreadsheet_name = ProGENI_rwr20_STExp_GDSC_500.rname.gxc.tsv phenotype_data_name = UCEC_phenotype.txt

---

⁠Description of Output files saved in results directory

---

• Output files of all four methods save genes by sample heatmap variances per row with name

genes_variance_{method}_{timestamp}_viz.tsv

• Output files of all four methods save genes by samples heatmap with name

genes_by_samples_heatmp_{method}_{timestamp}_viz.tsv

• Output files of all four methods save samples by samples heatmap with name

consensus_matrix_{method}_{timestamp}_viz.tsv

• Output files of all four methods save patients to cluster map with name

samples_labeled_by_cluster_{method}_{timestamp}_viz.tsv

• Output files of all four methods save gene scores by cluster with name

genes_averages_by_cluster_{method}_{timestamp}_viz.tsv

• Output files of all four methods save spreadsheet with top ranked genes per sample with name

top_genes_by_cluster_{method}_{timestamp}_download.tsv

• Output files of all four methods save patients to cluster map with name

phenotypes_labeled_by_cluster_{method}_{timestamp}_viz.tsv

• The clustering evaluation output file has the name

clustering_evaluation_result_{timestamp}.tsv