Docker image of the CBI version of the HCP Pipelines (v.3.4)
10K+
The HCP Pipelines product is a set of tools (primarily, but not exclusively, shell scripts) for processing MRI images for the [Human Connectome Project][HCP]. Among other things, these tools implement the Minimal Preprocessing Pipeline (MPP) described in [Glasser et al. 2013][GlasserEtAl]
The HCP Pipelines scripts are being released in essentially the form that they were successfully used for processing data provided in the [Human Connectome Project][HCP] 500 Subjects Data Release. That processing was done using 64-bit Red Hat Linux version 5.
Some improvements to the documentation have been made since they were used for that release.
One known difference between these scripts as they were used for the
500 Subjects Data Release and their currently released form is the version of the
wb_command from the [Connectome Workbench][Connectome-Workbench] that is used.
When these scripts were being used for the 500 Subjects Data Release, Connectome
Workbench had not reached its 1.0 release level. Therefore, we used an earlier
version of the wb_command for processing data for the data release. Distributing
that older version of the wb_command binary as part of this product is less than
ideal because that embedded binary is platform specific and is not
part of an official Connectome Workbench release.
If needed, the Red Hat/CentOS specific version of the wb_command binary and it
associated libraries used are available in older, tagged, pre-release versions of this
project.
We have begun the process of making the scripts more robust, well-documented, and usable in other environments (other versions of operating systems, other queueing systems, other versions of prerequisite tools, etc.) But that process is not complete. Therefore, we cannot guarantee that you will not have to make modifications to the scripts for them to be used successfully in your environment.
A significant part of the value of the open source model of software development is the ability to improve software based on source code level feedback from the community of users. Along those lines, if you find that you do need to make changes to the source code to use the tools successfully, we would welcome feedback and improvement suggestions. (If you have to change something in the scripts to make them work for you, let us know and we'll evaluate how/if to incorporate those changes into the released product.)
Discussion of HCP Pipeline usage and improvements can be posted to the hcp-users discussion list. Sign up for hcp-users at http://humanconnectome.org/contact/#subscribe
For the HCP 500 Subjects Data Release, FSL version 5.0.6 was used with these scripts. There is a known issue with using FSL 5.0.7 for the Task fMRI Analysis Pipeline. See the Prerequisites section below for further information.
Improvements to the internal documentation of these scripts are planned.
Validation tests to ensure that your installation is working correctly with at least the sample data are also planned but not yet available.
The HCP Pipelines have the following software requirements:
A 64-bit Linux Operating System
The [FMRIB Software Library][FSL] (a.k.a. [FSL][FSL]) version 5.0.6 installed and configuration file properly sourced.
NB: This version of the HCP Pipelines requires version 5.0.6 of FSL, not version 5.0.6 or greater. This version of the HCP Pipelines is not fully tested with the any version of FSL other than version 5.0.6. Preliminary testing has detected that there is a difference in behavior between version 5.0.6 of FSL and version 5.0.7 of FSL which, while it is an intentional improvement to FSL, is known to cause the Task Analysis pipeline in particular to fail.
There is currently a separate branch in this repository named fsl-5.0.7-changes.
That branch is not yet included in a released version of the code, but it contains
changes to the Task Analysis pipeline that we expect will fix that pipeline
so that it works with version 5.0.7 of FSL. These changes are not fully tested,
but they are available to anyone who wants to run the Task Analysis pipeline
and use FSL 5.0.7.
[FreeSurfer][FreeSurfer] version 5.3.0-HCP available at ftp://surfer.nmr.mgh.harvard.edu/pub/dist/freesurfer/5.3.0-HCP
NB: You must create and install a license file for FreeSurfer by visiting and submitting the FreeSurfer registration form.
NB: The version of FreeSurfer used is a special release of FreeSurfer which is not
part of the normal release cycle. The 5.3.0-HCP version of FreeSurfer contains a
slightly different version of the mris_make_surfaces program than is part
of the standard FreeSurfer 5.3.0 release.
[Connectome Workbench][Connectome-Workbench] version 1.0
The HCP Pipelines scripts use the wb_command which is part of the Connectome Workbench.
They locate the wb_command using an environment variable. Instructions for setting
this environment variable are provided below in the
Running the HCP Pipelines on example data section.
The [HCP version of gradunwarp][HCP-gradunwarp] version 1.0.2 (if gradient nonlinearity correction is to be done.)
Gradient Nonlinearity Correct is sometimes also referred to as Gradient Distortion Correction or GDC
As is true of the other prerequisite pieces of software, the HCP version of gradunwarp has its own set of prerequisites. See the HCP gradunwarp README file for those prerequisites.
In order to run HCP gradunwarp, you will need a gradient coefficients file to use as an input to the gradient distortion correction process. Please see questions 7 and 8 in the HCP Pipelines FAQ for further information about gradient nonlinearity correction and obtaining a gradient coefficients file.
The HCP Pipelines scripts expect to be able to find the main module of the
gradunwarp tool (gradient_unwarp.py) within a directory specified in the
PATH environment variable.
As distributed, the examples scripts that serve as templates for running
various types of pipeline processing are set to not run gradient
distortion correction. Commented out portions of those scripts illustrate
how to change the variable settings to perform gradient distortion correction.
These commented out portions assume that you have placed the gradient
coefficients file in the standard configuration directory for your
installation of HCP Pipelines (the global/config directory within
your HCP Pipelines installation directory.
Install the listed prerequisites first.
Installation Notes for FSL
Once you have installed FSL, verify that you have the correct version of FSL
by simply running the $ fsl command. The FSL window that shows
up should identify the version of FSL you are running in its title bar.
Sometimes FSL is installed without the separate documentation package, it is most likely worth the extra effort to install the FSL documentation package.
Ubuntu Installation Notes for FreeSurfer
For Linux, FreeSurfer is distributed in gzipped tarballs for CentOS 4 and CentOS 6.
The instructions here provide guidance for installing FreeSurfer on Ubuntu. If following the instructions there, be sure to download version 5.3.0-HCP of FreeSurfer and not version 5.1.0 as those instructions indicate.
Ubuntu (at least starting with version 12.04 and running through version 14.04 LTS)
is missing a library that is used by some parts of FreeSurfer. To install
that library enter $ sudo apt-get install libjpeg62.
Download the necessary compressed tar file (.tar.gz) for the [HCP Pipelines release][HCP-pipelines-release].
Move the compressed tar file that you download to the directory in which you want the HCP Pipelines to be installed, e.g.
$ mv Pipelines-3.4.0.tar.gz ~/projects
Extract the files from the compressed tar file, e.g.
$ cd ~/projects
$ tar xvf Pipelines-3.4.0.tar.gz
This will create a directory containing the HCP Pipelines, e.g.
$ cd ~/projects/Pipelines-3.4.0
$ ls -F
DiffusionPreprocessing/ fMRIVolume/ PostFreeSurfer/ TaskfMRIAnalysis/
Examples/ FreeSurfer/ PreFreeSurfer/ tfMRI/
FAQ.md global/ product.txt VersionHistory.md*
fMRISurface/ LICENSE.md* README.md version.txt
$
This newly created directory is your HCP Pipelines Directory.
In this documentation, in documentation within the script files themselves,
and elsewhere, we will use the terminology HCP Pipelines Directory
interchangably with HCPPIPEDIR, $HCPPIPEDIR, or ${HCPPIPEDIR}.
More specifically, $HCPPIPEDIR and ${HCPPIPEDIR} refer to an environment
variable that will be set to contain the path to your HCP Pipelines Directory.
Example data for becoming familiar with the process of running the HCP Pipelines and testing your installation is available from the Human Connectome Project.
If you already have (or will be obtaining) the gradient coefficients file for the Connectome Skyra scanner used to collect the sample data and want to run the pipelines including the steps which perform gradient distortion correction, you can download a zip file containing example data here.
In that case, you will need to place the obtained gradient coefficients
file (coeff_SC72C_Skyra.grad) in the global/config directory within
your HCP Pipelines Directory.
If you do not have and are not planning to obtain the gradient coefficients file for the Connectome Skyra scanner used to collect the sample data and want to run the pipelines on files on which gradient distortion correction has already been performed, you should should download a zip file containing example data here.
The remainder of these instructions assume you have extracted the example data
into the directory ~/projects/Pipelines_ExampleData. You will need to
modify the instructions accordingly if you have extracted the example data
elsewhere.
Structural preprocessing is subdivided into 3 parts (Pre-FreeSurfer processing,
FreeSurfer processing, and Post-FreeSurfer processing). These 3 steps should
be excuted in the order specified, and each of these 3 parts is implemented
as a separate bash script.
In the ${HCPPIPEDIR}/Examples/Scripts directory, you will find
a shell script for running a batch of subject data through the
Pre-FreeSurfer part of structural preprocessing. This shell script is
named: PreFreeSurferPipelineBatch.sh. You should review and possibly
edit that script file to run the example data through the
Pre-FreeSurfer processing.
StudyFolder
The setting of the StudyFolder variable near the top of this script
should be verified or edited. This variable should contain the path to a
directory that will contain data for all subjects in subdirectories named
for each of the subject IDs.
As distributed, this variable is set with the assumption that you have
extracted the sample data into a directory named projects/Pipelines_ExampleData
within your login or "home" directory.
StudyFolder="${HOME}/projects/Pipelines_ExampleData"
You should either verify that your example data is extracted to that location or modify the variable setting accordingly.
Subjlist
The setting of the Subjlist variable, which comes immediately
after the setting of the StudyFolder variable, should also be
verified or edited. This variable should contain a space delimited
list of the subject IDs for which you want the Pre-FreeSurfer processing
to run.
As distributed, this variable is set with the assumption that you will
run the processing only for the single example subject, which has a
subject ID of 100307.
Subjlist="100307"
Using this value in conjunction with the value of the StudyFolder variable,
the script will look for a directory named 100307 within the directory
${HOME}/projects/Pipelines_ExampleData. This is where it will expect to
find the data it is to process.
You should either verify that your example data is in that location or modify the variable setting accordingly.
EnvironmentScript
The EnvironmentScript variable should contain the path to a
script that sets up the environment variables that are necessary
for running the Pipeline scripts.
As distributed, this variable is set with the assumption that you
have installed the HCP Pipelines in the directory
${HOME}/projects/Pipelines (i.e. that your HCP Pipelines directory
is ${HOME}/projects/Pipelines) and that you will use the
example environment setup provided in the
Examples/Scripts/SetUpHCPPipeline.sh script.
You may need to update the setting of the EnvironmentScript
variable to reflect where you have installed the HCP Pipelines.
GradientDistortionCoeffs
Further down in the script, the GradientDistortionCoeffs variable
is set. This variable should be set to contain either the path to
the gradient coefficients file to be used for gradient distortion
correction or the value NONE to skip over the gradient distortion
correction step.
As distributed, the script sets the variable to skip the gradient distortion correction step.
You will need to update the setting of this variable if you have a gradient coefficients file to use and want to perform the gradient distortion correction
HCPPIPEDIR and the SetUpHCPPipeline.sh script
The script file referenced by the EnvironmentScript variable
in the PreFreeSurferPipelineBatch.sh file (by default the
SetUpHCPPipeline.sh file in the Examples\Scripts folder)
does nothing but establish values for all the environment
variables that will be needed by various pipeline scripts.
Many of the environment variables set in the SetUpHCPPipeline.sh
script are set relative to the HCPPIPEDIR environment variable.
As distributed, the setting of the HCPPIPEDIR environment
variable assumes that you have installed the HCP Pipelines
in the ${HOME}/projects/Pipelines directory. You may
need to change this to reflect your actual installation
directory.
As distributed, the SetUpHCPPipeline.sh script assumes
that you have:
Example statements for setting FSLDIR, sourcing the
FSL configuration script, setting FREESURFER_HOME,
and sourcing the FreeSurfer setup script are provided
but commented out in the SetUpHCPPipeline.sh script
prior to setting HCPPIPEDIR.
CARET7DIR in the SetUpHCPPipeline.sh script
The CARET7DIR variable must provide the path to the directory
in which to find the Connectome Workbench wb_command. As
distributed, the CARET7DIR is set with the assumption that
the necessary wb_command binary is installed in the
${HOME}/workbench/bin_linux64 directory.
It is very likely that you will need to change the value of the
CARET7DIR environment variable to indicate the location of your
installed version of the wb_command.
Running the Pre-FreeSurfer processing after editing the setup script
Once you have made any necessary edits as described above, Pre-FreeSurfer processing can be invoked by commands similar to:
$ cd ~/projects/Pipelines/Examples/Scripts
$ ./PreFreeSurferPipelineBatch.sh
This script must be SOURCED to correctly setup the environment
prior to running any of the other HCP scripts contained here
100307
Found 1 T1w Images for subject 100307
Found 1 T2w Images for subject 100307
About to use fsl_sub to queue or run /home/user/projects/Pipelines/PreFreeSurfer/PreFreeSurferPipeline.sh
After reporting the number of T1w and T2w images found, the
PreFreeSurferPipelineBatch.sh script uses the FSL command fsl_sub
to submit a processing job which ultimately runs the PreFreeSurferPipeline.sh
pipeline script.
If your system is configured to run jobs via an Oracle Grid Engine cluster
(previously known as a Sun Grid Engine (SGE) cluster), then fsl_sub will
submit a job to run the PreFreeSurferPipeline.sh script on the
cluster and then return you to your system prompt. You can check on the
status of your running cluster job using the qstat command. See the
documentation of the qstat command for further information.
The standard output (stdout) and standard error (stderr) for the job
submitted to the cluster will be redirected to files in the directory
from which you invoked the batch script. Those files will be named
PreFreeSurferPipeline.sh.o<job-id> and
PreFreeSurferPipeline.sh.e<job-id> respectively, where
<job-id> is the cluster job ID. You can monitor the progress
of the processing with a command like:
$ tail -f PreFreeSurferPipeline.sh.o1434030
where 1434030 is the cluster job ID.
If your system is not configured to run jobs via an Oracle Grid Engine
cluster, fsl_sub will run the PreFreeSurferPipeline.sh script
directly on the system from which you launched the batch script.
Your invocation of the batch script will appear to reach a point
at which "nothing is happening." However, the PreFreeSurferPipeline.sh
script will be launched in a separate process and the
standard output (stdout) and standard error (stderr) will have
been redirected to files in the directory from which you invoked
the batch script. The files will be named
PreFreeSurferPipeline.sh.o<process-id> and
PreFreeSurferPipeline.sh.e<process-id>' respectively, where ` is the operating system assigned unique process
ID for the running process.
A similar tail command to the one above will allow you to monitor
the progress of the processing.
Keep in mind that depending upon your processor speed and whether or not you are performing gradient distortion correction, the Pre-FreeSurfer phase of processing can take several hours.
In the ${HCPPIPEDIR}/Examples/Scripts directory, you will find
a shell script for running a batch of subject data through the
FreeSurfer part of structural preprocessing. This shell script is
named: FreeSurferPipelineBatch.sh. You should review and possibly
edit that script file to run the example data through the
FreeSurfer processing.
The StudyFolder, Subjlist, and EnvironmentScript variables
are set near the top of the script and should be verified and edited
as indicated above in the discussion of Pre-FreeSurfer processing.
Your environment script (SetUpHCPPipeline.sh) will need to have
the same environment variables set as for the Pre-FreeSurfer
processing.
Once you have made any necessary edits as described above and the Pre-FreeSurfer processing has completed, then invoking FreeSurfer processing is quite similar to invoking Pre-FreeSurfer processing. The command will be similar to:
$ cd ~/projects/Pipelines/Examples/Scripts
$ ./FreeSurferPipelineBatch.sh
This script must be SOURCED to correctly setup the environment
prior to running any of the other HCP scripts contained here
100307
About to use fsl_sub to queue or run /home/user/projects/Pipelines/FreeSurfer/FreeSurferPipeline.sh
As above, the fsl_sub command will either start a new process on the current
system or submit a job to an Oracle Grid Engine cluster. Also as above, you can
monitor the progress by viewing the generated standard output and standard error
files.
In the ${HCPPIPEDIR}/Examples/Scripts directory, you will find
a shell script for running a batch of subject data through the
Post-FreeSurfer part of structural preprocessing. This shell script
is named: PostFreeSurferPipelineBatch.sh. This script follows
the same pattern as the batch scripts to run Pre-FreeSurfer and
FreeSurfer processing do. That is, you will need to verify/edit
the StudyFolder, Subjlist, and EnvironmentScript variables
that are set at the top of the script, and your environment
script (SetUpHCPPipeline.sh) will need to set environment variables
appropriately.
There is an additional variable in the PostFreeSurferPipelineBatch.sh
script that needs to be considered. The RegName variable tells
the pipeline whether to use MSMSulc for surface alignment. (See
the [FAQ][FAQ] for further information about MSMSulc.) As distributed,
the PostFreeSurferPipelineBatch.sh script assumes that you do not
have access to the msm binary to use for surface alignment. Therefore,
the RegName variable is set to "FS".
If you do have access to the msm binary and wish to use it, you
can change this variable's value to "MSMSulc". If you do so, then
your environment script (e.g. SetUpHCPPipeline.sh) will need to set
an additional environment variable which is used by the pipeline
scripts to locate the msm binary. That environment variable is
MSMBin and it is set in the distributed example SetUpHCPPipeline.sh
file as follows:
export MSMBin=${HCPPIPEDIR}/MSMBinaries
You will need to either place your msm executable binary file
in the ${HCPPIPEDIR}/MSMBinaries directory or modify the
value given to the MSMBin environment variable so that it
contains the path to the directory in which you have placed
your copy of the msm executable.
Once those things are taken care of and FreeSurfer processing is completed, commands like the following can be issued:
$ cd ~/projects/Pipelines/Examples/Scripts
$ ./PostFreeSurferPipelineBatch.sh
This script must be SOURCED to correctly setup the environment
prior to running any of the other HCP scripts contained here
100307
About to use fsl_sub to queue or run /home/user/projects/Pipelines/PostFreeSurfer/PostFreeSurferPipeline.sh
The fsl_sub command used in the batch script will behave as described
above and monitoring the progress of the run can be done as described
above.
Diffusion Preprocessing depends on the outputs generated by Structural Preprocessing. So Diffusion Preprocessing should not be attempted on data sets for which Structural Preprocessing is not yet complete.
The DiffusionPreprocessingBatch.sh script in the ${HCPPIPEDIR}/Examples/Scripts
directory is much like the example scripts for the 3 phases of Structural
P
Content type
Image
Digest
Size
2.5 GB
Last updated
over 7 years ago
docker pull cbinyu/pipelines