Public Repository

Last pushed: a month ago
Short Description
Next-gen pathway (iPathwayGuide) and variant analyses (iVariantGuide).
Full Description

Advaita Cloud API client

Prerequisites

Docker version 1.8 or later. https://www.docker.com

Installation

Download the Docker image using the following command:

docker pull advaitabio/api-client

This will allow you to:

First, obtain your API credentials from https://apps.advaitabio.com/oauth-provider/profile. For all examples below, we will consider the credentials to be:

Client Id:      62cf60f5-b7dd-4c57-8158-ab79e7d1a41d
Client Secret:  BsVRJSmQpB70Xru1IpJV6C66ECbAi47E

Second, download example data:

  • GSE52194: mRNA-sequencing of breast cancer subtypes and normal tissue (the TNBC1 sample)

    • you can find more information about the experiment on the GEO website
    • download the VCF file from this link
  • GSE37703: Differential analysis of HOXA1 in adult cells at isoform resolution by RNA-Seq [Illumina]

    • you can find more information about the experiment on the GEO website
    • download the CUFFDIFF file from this link

The following commands will create a folder and download the above two samples:

$ mkdir -p ~/experiments
$ curl -o ~/experiments/ipg-example-diff-hu-GSE37703_hiseq_cuffdiff.diff https://s3-us-west-2.amazonaws.com/advaitabio/public/example_data/ipg-example-diff-hu-GSE37703_hiseq_cuffdiff.diff
$ curl -o ~/experiments/ivg-example-hg19-ss-gsm1261016.vcf.gz https://s3-us-west-2.amazonaws.com/advaitabio/public/example_data/ivg-example-hg19-ss-gsm1261016.vcf.gz

Submitting variant analyses to iVariantGuide

After obtaining the client credentials and example data you can submit a variant analysis by running the following command:

$ docker run -v ~/experiments/:/data/ \
    -e 'CLIENT_ID=62cf60f5-b7dd-4c57-8158-ab79e7d1a41d' \
    -e 'CLIENT_SECRET=BsVRJSmQpB70Xru1IpJV6C66ECbAi47E' \
    -e 'INPUT_FILE_PATH=/data/ivg-example-hg19-ss-gsm1261016.vcf.gz' \
    advaitabio/api-client ivg-client
/usr/bin/ivg-client: Getting access token...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   111    0   111    0     0    161      0 --:--:-- --:--:-- --:--:--   161

...

/usr/bin/ivg-client: Status is COMPLETED
/usr/bin/ivg-client: Output file written to /data/ivg-example-hg19-ss-gsm1261016.vcf.gz_03585bbc-1221-42c5-a907-4ece0e91db85.json
/usr/bin/ivg-client: HTML Output file written to /data/ivg-example-hg19-ss-gsm1261016.vcf.gz_03585bbc-1221-42c5-a907-4ece0e91db85.b64html

Using this command the client will check periodically the satus of the analysis and it will stop when the analysis completes. If you want to submit the analysis without following the status see section Checking status of a report manually below.

As an alternative, if you want to use the API client command line directly, you can start a docker container in interactive mode like this:

$ docker run -v ~/experiments/:/data/ -it advaitabio/api-client /bin/bash

Once the container has started, you can view the help by running the ivg-client --help command line.

[root@38f8e71a0d5f tmp]# ivg-client --help
usage: ivg-client [-c <client-id>] [-s <client-secret>] [-f <input-file>]
 [-a <assembly>] [-o output-file] [-m html-output-file]
 [-t <title>] [-d <description>] [-D <do-not-follow>] [-A <analysis-request-id>] [-h]

where:
    -c,--client-id            api client id
    -s,--client-secret        api client secret
    -f,--input-file           file to analyze
    -a,--assembly             assembly
    -o,--output-file          output file
    -m,--html-output-file     html output file
    -t,--title                report title
    -d,--description          report description
    -D,--do-not-follow        do not wait for report to be completed
    -A,--analysis-request-id  id of report submitted
    -h,--help                 show this help text

To execute the same variant analysis as above, you can run:

[root@38f8e71a0d5f tmp]# ivg-client -c 62cf60f5-b7dd-4c57-8158-ab79e7d1a41d \
      -s BsVRJSmQpB70Xru1IpJV6C66ECbAi47E -f /data/ivg-example-hg19-ss-gsm1261016.vcf.gz
/usr/bin/ivg-client: Getting access token...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   111    0   111    0     0    161      0 --:--:-- --:--:-- --:--:--   161

...

/usr/bin/ivg-client: Status is COMPLETED
/usr/bin/ivg-client: Output file written to /data/ivg-example-hg19-ss-gsm1261016.vcf.gz_03585bbc-1221-42c5-a907-4ece0e91db85.json
/usr/bin/ivg-client: HTML Output file written to /data/ivg-example-hg19-ss-gsm1261016.vcf.gz_03585bbc-1221-42c5-a907-4ece0e91db85.b64html

Note: Environment variables are considered when running the API client in the command line. However, the command line arguments take priority.

Submitting gene expression analyses to iPathwayGuide

After obtaining the client credentials and example data you can submit a pathway analysis by running the following command:

$ docker run -v ~/experiments/:/data/ \
    -e 'CLIENT_ID=62cf60f5-b7dd-4c57-8158-ab79e7d1a41d' \
    -e 'CLIENT_SECRET=BsVRJSmQpB70Xru1IpJV6C66ECbAi47E' \
    -e 'INPUT_FILE_PATH=/data/ipg-example-diff-hu-GSE37703_hiseq_cuffdiff.diff' \
    advaitabio/api-client ipg-client
/usr/bin/ipg-client: Getting access token...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   111    0   111    0     0    161      0 --:--:-- --:--:-- --:--:--   161

...

/usr/bin/ipg-client: Status is COMPLETED
/usr/bin/ipg-client: Output file written to /data/ipg-example-diff-hu-GSE37703_hiseq_cuffdiff.diff_03585bbc-1221-42c5-a907-4ece0e91db85.json
/usr/bin/ipg-client: HTML Output file written to /data/ipg-example-diff-hu-GSE37703_hiseq_cuffdiff.diff_03585bbc-1221-42c5-a907-4ece0e91db85.b64html

Using this command the client will check periodically the satus of the analysis and it will stop when the analysis completes. If you want to submit the analysis without following the status see section Checking status of a report manually below.

As an alternative, if you want to use the API client command line directly, you can start a docker container in interactive mode like this:

$ docker run -v ~/experiments/:/data/ -it advaitabio/api-client /bin/bash

Once the container has started, you can view the help by running the ipg-client --help command line.

[root@38f8e71a0d5f tmp]# ipg-client --help 
usage: ipg-client [-c <client-id>] [-s <client-secret>] [-f <input-file>]
 [-o output-file] [-m html-output-file]
 [-t <title>] [-d <description>] [-O <organism>] [-p <adjpv>] 
 [-l <logfc>] [-F <file-type>] [-D <do-not-follow>] [-A <analysis-request-id>] [-h]

where:
    -c,--client-id            api client id
    -s,--client-secret        api client secret
    -f,--input-file           file to analyze
    -F,--file-type            type of file
    -o,--output-file          output file
    -m,--html-output-file     html output file
    -O,--organism             organism
    -p,--adjpv                adjusted p-value
    -l,--logfc                log fold change 
    -t,--title                report title
    -d,--description          report description
    -D,--do-not-follow        do not wait for report to be completed
    -A,--analysis-request-id  id of report submitted
    -h,--help                 show this help text

To execute the same expression analysis as above, you can run:

[root@38f8e71a0d5f tmp]# ipg-client -c 62cf60f5-b7dd-4c57-8158-ab79e7d1a41d \
      -s BsVRJSmQpB70Xru1IpJV6C66ECbAi47E -f /data/ipg-example-diff-hu-GSE37703_hiseq_cuffdiff.diff
/usr/bin/ipg-client: Getting access token...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   111    0   111    0     0    161      0 --:--:-- --:--:-- --:--:--   161

...

/usr/bin/ipg-client: Status is COMPLETED
/usr/bin/ipg-client: Output file written to /data/ipg-example-diff-hu-GSE37703_hiseq_cuffdiff.diff_03585bbc-1221-42c5-a907-4ece0e91db85.json
/usr/bin/ipg-client: HTML Output file written to /data/ipg-example-diff-hu-GSE37703_hiseq_cuffdiff.diff_03585bbc-1221-42c5-a907-4ece0e91db85.b64html

Note: Environment variables are considered when running the API client in the command line. However, the command line arguments take priority.

Configuration parameters

The client is configured using environment variables that are passed using the -e option to the docker run command. Mandatory parameters:

  • CLIENT_ID: The value of API ID from API credentials https://apps.advaitabio.com/oauth-provider/profile

  • CLIENT_SECRET: The value of API Secret from API credentials https://apps.advaitabio.com/oauth-provider/profile

  • INPUT_FILE_PATH: Path relative to the container where the input file was mounted using -v. To ease the configuration we recommend mounting the whole folder where the input file resides. If you wish to mount only the input file the output will not be available on the host machine (see OUTPUT_FILE_PATH)

iVariantGuide optional parameters
  • ASSEMBLY (optional, default value: hg19): Reference genome corresponding to the input file as specified in INPUT_FILE_PATH. Available options are: hg19 and hg38.
iPathwayGuide optional parameters
  • FILE_TYPE (optional, default value: CUFFDIFF): The type of the input file as specified in INPUT_FILE_PATH. Available options are: CUFFDIFF and DESEQ. The files must be decompressed.

  • ORGANISM (optional, default value is human taxonomy id: 9606): Organism taxonomy id corresponding to the input file. Available options are: 9606, 10090 and 10116.

  • ADJPV (optional, default value: 0.05): Adjusted p-value.

  • LOGFC (optional, default value: 0.6): Log fold change.

Common optional parameters
  • OUTPUT_FILEPATH (optional, default value: ``{inputfilename}{universal_unique_identifier}.json`` ): Path relative to the container where the output file is saved.

  • TITLE (optional, default value: api-proxy-report): The title of the analysis. Use only alphanumeric character, -, _ or space.

  • DESCRIPTION (optional, default value:created with api-proxy version v1): A brief description of the report. Expecting one line of text without quotes

  • CURL_OPTS (optional -debug level, default value: --retry 5): CURL options (e.g.: --silent - no curl logs, -v - verbose, etc.)

  • LOGGER_OPTS (optional, default value -s -t $0): the logs are saved to error output, the tag is the name of the script. For more options consult the logger manual

  • DO_NOT_FOLLOW (optional, default value: 0): Available options are: 0 and 1. This flag will stop the script as soon as the report is submitted. It will not wait to check the status of the report. This means the status of the report must be checked manually.

  • ANALYSIS_REQUEST_ID (optional): Id of the report submitted. This can be used when checking the status of the report manually.

Checking status of a report manually

Whether you are running interactively or non-interactively, there is an option to check the status of your report manually instead of waiting for the script to finish.

To execute a variant analysis with DO_NOT_FOLLOW flag on in interactive mode, you can run:

[root@38f8e71a0d5f tmp]# ivg-client -c 62cf60f5-b7dd-4c57-8158-ab79e7d1a41d \
      -s BsVRJSmQpB70Xru1IpJV6C66ECbAi47E -f /data/ivg-example-hg19-ss-gsm1261016.vcf.gz -D
/usr/bin/ivg-client: Getting access token...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   111    0   111    0     0    161      0 --:--:-- --:--:-- --:--:--   161

...

/usr/bin/ivg-client: POST-ed successfully to https://api.advaitabio.com/v1b1/analysisRequests
/usr/bin/ivg-client: Analysis-Request-Id is a9282d5f-cd6d-4e46-9f59-6a0bb6992e5c
/usr/bin/ivg-client: Analysis request with id 'a9282d5f-cd6d-4e46-9f59-6a0bb6992e5c' was created. To check the status, run the command again by providing the analysis request id instead of the input file.

Since we are running interactive mode, we will use command line parameters (e.g., use -A to pass the analysis identifier obtained in the previous step).
To check the status of your report, you can run:

[root@38f8e71a0d5f tmp]# ivg-client -c 62cf60f5-b7dd-4c57-8158-ab79e7d1a41d -s
      BsVRJSmQpB70Xru1IpJV6C66ECbAi47E -A a9282d5f-cd6d-4e46-9f59-6a0bb6992e5c'
/usr/bin/ivg-client: Getting access token...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   111    0   111    0     0    190      0 --:--:-- --:--:-- --:--:--   190

...

/usr/bin/ivg-client: Status is COMPLETED
/usr/bin/ivg-client: Output file written to /tmp/a9282d5f-cd6d-4e46-9f59-6a0bb6992e5c.json
/usr/bin/ivg-client: HTML Output file written to /tmp/a9282d5f-cd6d-4e46-9f59-6a0bb6992e5c.b64html

Note: If you want to specify where the output files will be written, you can do so by setting OUTPUT_FILE_PATH environment variable in non-interactive mode, and -o parameter in interactive mode. The default path is
/tmp/{Analysis-Request-Id}.json

Known limitations

{
  "errorMessage": "Error: Could not upload file",
  "httpStatus": "500",
  "httpResponse": [
    {
      "timestamp": 1479840779204,
      "status": 500,
      "error": "Internal Server Error",
      "exception": "com.netflix.zuul.exception.ZuulException",
      "message": "TIMEOUT"
    }
  ]
}
  • You may only submit one file at a time. Each file must contain a single-sample. All other requests will result in an ERROR status. Here is an example of request that ends with status error due to multi-sample input file:
{
  "output" : {
    "fileCheck" : {
      "id" : "e3db114b-14c2-4cb1-9e07-c9330ff430db",
      "statusMessages" : [ {
        "message" : "The file exceeds the current uploading limit (1 samples per file)."
      } ],
      "status" : "ERROR",
      "metadata" : {
        "assembly" : "hg19",
        "originalFileName" : "testdata.vcf",
        "originalFileId" : "cd79a21d-3310-4528-b3a9-64214f78e3ee"
      }
    }
  },
  "status" : "ERROR",
  "input" : {
    "fileId" : "cd79a21d-3310-4528-b3a9-64214f78e3ee"
  },
  "creationTime" : "2016-11-22T18:30:31.311+0000",
  "id" : "0d32ce8d-12ad-4b50-a2f5-34267807773b",
  "type" : "VARIANTS_SINGLE_SAMPLE",
  "parameters" : {
    "title" : "api-proxy-report",
    "description" : "created with api-proxy version 1",
    "assembly" : "hg19",
    "organism" : 9606
  },
  "_links" : {
    "self" : {
      "href" : "https://api.advaitabio.com:443/v1b1/analysisRequests/0d32ce8d-12ad-4b50-a2f5-34267807773b"
    }
  }
}

Release Notes

2.0.0
  • added independent clients for submitting iVariantGuide and iPathwayGuide analyses
1.1.0
  • submit gene expression analyses to iPathwayGuide (CuffDiff and DESeq)
  • added "DO_NOT_FOLLOW" option to manually check the status of the report for both iPathwayGuide and iVariantGuide
  • bugfix: fixed the issue when uploading VCF files in non-interactive mode
1.0.2
  • added interactive mode
1.0.0
  • submit variant analyses to iVariantGuide

For all other errors, consult the frequently asked questions for iVariantGuide and frequently asked questions for iPathwayGuide pages or contact us and provide the output error file.

(C) Advaita Bioinformatics, All Rights Reserved

Docker Pull Command
Owner
advaitabio