created by Geraldine_VdAuwera
on 2016-02-26
Install all software packages required to attend a GATK workshop.
To follow these instructions, you will need to have a basic understanding of the meaning of the following words and command-line operations. If you are unfamiliar with any of the following, you should consult a more experienced colleague or your system administrator if you have one. There are also many good online tutorials you can use to learn the necessary notions.
GATK is supported on all flavors of reasonably recent Linux/Unix and MacOS X systems, but NOT on Windows. The analyses we run in workshops are designed to run quickly and on small datasets, so should not require more than 2G of RAM. For file storage, plan on 10G of space (but I would be shocked if we get to half of that).
The current version of GATK requires Java Runtime Environment version 1.8. All Linux/Unix and MacOS X systems should have a JRE pre-installed, but the version may vary. To test your Java version, run the following command in the shell:
java -version
This should return a message along the lines of ”java version 1.8.0_65” as well as some details on the Runtime Environment (JRE) and Virtual Machine (VM). If you have a version other than 1.8.x, be aware that you may run into trouble with some of the more advanced features of the Picard and GATK tools. The simplest solution is to install an additional JRE and specify which you want to use at the command-line. To find out how to do so, you should seek help from your system administrator and read this article.
Read the overview of the Picard software on the Picard project homepage, then download the latest version (currently 2.4.1) of the package containing the pre-compiled program file (the picard-tools-2.x.y.zip file).
Unpack the zip file using:
tar xjf picard-tools-2.4.1.zip
This will produce a directory called picard-tools-2.4.1 containing the Picard jar files. Picard tools are distributed as a pre-compiled Java executable (jar file) so there is no need to compile them.
Note that it is not possible to add jar files to your path to make the tools available on the command line; you have to specify the full path to the jar file in your java command, which would look like this:
java -jar ~/my_tools/jars/picard.jar <Toolname> [options]
This syntax will be explained in a little more detail further below.
However, you can set up a shortcut called an "environment variable" in your shell profile configuration to make this easier. The idea is that you create a variable that tells your system where to find a given jar, like this:
PICARD = "~/my_tools/jars/picard.jar"
So then when you want to run a Picard tool, you just need to call the jar by its shortcut, like this:
java -jar $PICARD <Toolname> [options]
The exact way to set this up depends on what shell you're using and how your environment is configured. We like this overview and tutorial which explains how it all works; but if you are new to the command line environment and you find this too much too deal with, we recommend asking for help from your institution's IT support group.
This completes the installation process.
Open a shell and run:
java -jar picard.jar -h
This should print out some version and usage information about the AddOrReplaceReadGroups.jar tool. At this point you will have noticed an important difference between BWA and Picard tools. To use BWA, we called on the BWA program and specified which of its internal tools we wanted to apply. To use Picard, we called on Java itself as the main program, then specified which jar file to use, knowing that one jar file = one tool. This applies to all Picard tools; to use them you will always build your command lines like this:
java -jar picard.jar <ToolName> [options]
This means you first make the call to Java itself as the main program, then specify the picard.jar file, then specify which tool you want, and finally you pass whatever other arguments (input files, parameters etc.) are needed for the analysis.
Note that the command-line syntax of Picard tools has recently changed from java -jar <ToolName>.jar to java -jar picard.jar <ToolName>. We are using the newer syntax in this document, but some of our other documents may not have been updated yet. If you encounter any documents using the old syntax, let us know and we'll update them accordingly. If you are already using an older version of Picard, either adapt the commands or better, upgrade your version!
Next we will see that GATK tools are called in essentially the same way, although the way the options are specified is a little different. The reasons for how tools in a given software package are organized and invoked are largely due to the preferences of the software developers. They generally do not reflect strict technical requirements, although they can have an effect on speed and efficiency.
Hopefully if you're reading this, you're already acquainted with the purpose of the GATK, so go ahead and download the latest version of the software package.
In order to access the downloads, you need to register for a free account on the GATK support forum. You will also need to read and accept the license agreement before downloading the GATK software package. Note that if you intend to use the GATK for commercial purposes, you will need to purchase a license. See the licensing page for an overview of the commercial licensing conditions.
Unpack the tar file using:
tar xjf GenomeAnalysisTK-3.8-0.tar.bz2
This will produce a directory called GenomeAnalysisTK-3.8-0 containing the GATK jar file, which is called GenomeAnalysisTK.jar, as well as a directory of example files called resources. GATK tools are distributed as a single pre-compiled Java executable so there is no need to compile them. Just like we discussed for Picard, it's not possible to add the GATK to your path, but you can set up a shortcut to the jar file using environment variables as described above.
This completes the installation process.
Open a shell and run:
java -jar GenomeAnalysisTK.jar -h
This should print out some version and usage information, as well as a list of the tools included in the GATK. As the Usage line states, to use GATK you will always build your command lines like this:
java -jar GenomeAnalysisTK.jar -T <ToolName> [arguments]
This means that just like for Picard, you first make the call to Java itself as the main program, then specify the GenomeAnalysisTK.jar file, then specify which tool you want, and finally you pass whatever other arguments (input files, parameters etc.) are needed for the analysis.
If you're attending a workshop that will cover somatic variant analysis, you will need to get the GATK4 beta. Instructions are provided here. As of September 2017, the workshop tutorials use version 4.beta.4 specifically.
The Integrated Genomics Viewer is a genome browser that allows you to view BAM, VCF and other genomic file information in context. It has a graphical user interface that is very easy to use, and can be downloaded for free (though registration is required) from this website. We encourage you to read through IGV's very helpful user guide, which includes many detailed tutorials that will help you use the program most effectively.
Download the latest version of RStudio IDE. The webpage should automatically detect what platform you are running on and recommend the version most suitable for your system.
Follow the installation instructions provided. Binaries are provided for all major platforms; typically they just need to be placed in your Applications (or Programs) directory. Open RStudio and type the following command in the console window:
install.packages("ggplot2")
This will download and install the ggplot2 library as well as any other library packages that ggplot2 depends on for its operation. Note that some users have reported having to install two additional package themselves, called reshape and gplots, which you can do as follows:
install.packages("reshape") install.packages("gplots")
In some sessions we also use an additional utility package called gridExtra; go ahead and install it as follows:
install.packages("gridExtra")
Finally, do the same thing to install the gsalib library:
install.packages("gsalib")
This will download and install the gsalib library.
Read the overview of the SAMtools software on the SAMtools project homepage, then download the latest version of the software package.
Unpack the tar file using:
tar xvjf samtools-0.1.2.tar.bz2
This will produce a directory called samtools-0.1.2 containing the files necessary to compile the SAMtools binary. Move to this directory and compile using:
cd samtools-0.1.2 make
The compiled binary is called samtools. You should find it within the same folder (samtools-0.1.2 in this example). Finally, add the SAMtools binary to your path to make it available on the command line. This completes the installation process.
Open a shell and run:
samtools
This should print out some version information as well as a list of commands. As the Usage line states, to use SAMtools you will always build your command lines like this:
samtools <command> [options]
This means you first make the call to the binary (samtools), then you specify which command (method) you wish to use (e.g. index) then any options (i.e. arguments such as input files or parameters) used by the program to perform that command. This is a similar convention as used by BWA.
RTG Tools is a free open-source software package produced by a commercial company called Real Time Genomics. This toolkit includes some variant evaluation and plotting tools that we find useful for teaching because they're fairly user-friendly and produce neat interactive plots.
You can download the toolkit from the RTG website, which provides packages for Linux, MacOS X and Windows.
After unzipping the file, follow the instructions in the README file that's included in the download package. On a Mac, moving the package to your preferred location and adding the rtg binary to your path to make it available on the command line is sufficient to complete the installation process.
Open a shell and run:
rtg
This should print out some usage information as well as a list of commands. As stated, to use the RTG tools you will always build your command lines like this:
rtg <command> [options]
This means you first make the call to the binary (rtg), then you specify which command (method) you wish to use (e.g. vcfeval) then any options (i.e. arguments such as input files or parameters) used by the program to perform that command. This is a similar convention as used by BWA.
We will use RTG Tools’s modules vcfeval and rocplot. You'll find a PDF file named RTGOperationsManual.pdf containing detailed documentation included in the download packge. For our workshops, the relevant pages are pages 38–42 (for vcfeval) and pages 44–46 (for rocplot).
If you're attending a workshop that will cover pipelining with WDL, you will need to get the jar files for the Cromwell execution engine and a utility called WDLTool. We also recommend a text editor called SublimeText. Instructions are provided here.
Updated on 2017-09-10
From shlee on 2017-09-09
For the Variant Filtering tutorial as of November, 2016, be sure to install the following five R packages in RStudio:
```
gsalib, ggplot2, gridExtra, reshape, gplots
```
Here’s how:
