Using Evoker with Broad Data

Downloading Intensity Files

To obtain the raw intensity files, you need to download all of the .gtc.txt files for a given project. This can be done using the Download tool available from the GAP portal website.

1. Download the zipped shell script:

wget https://www.broadinstitute.org/portal/GAPPortal/public/files/download-portal-2.7.zip

2. Unzip the file.

unzip download-portal-2.7.zip

3. Run the shell script as follows:

sh download-portal-data.sh -u my.username -s studyid -d directory

where my.username is your GAP portal login ID, studyid is the studyid for the data you want to download available on the study workspace page, and directory is the directory you want the files to be downloaded to. I strongly encourage you to use /broad/hptmp/ for this purpose!!! This data can be up to 1 TB in size with many small files.

4. Enter your password when prompted.

5. Once you see files downloading, then hit ctrl-c to quit the download.

6. There will be a text file (portalfilelist.txt) with a list of files that will be downloaded. To avoid downloading unnecessary files, do something similar to the following:

grep "name\|.gtc.txt\|true" portalfilelist.txt | grep -v "egt\|idat\|bscalls" > mydata2download

7. Rerun the original shell download command, but this time adding an extra argument -f pointing to the file with the list of filenames to download created in step 6.

sh download-portal-data.sh -u my.username -s studyid -d directory -f mydata2download

8. Enter your password again, and this time only the .gtc.txt files will be downloaded.

Creating an Evoker Project

A set of Evoker files consists of a binary PLINK bed and bim files that are split by chromosome and a corresponding binary file of intensity values. There is one fam file that does not have a chromosome number in the file name that gives the IDs for all files. An example completed project looks as follows:

russian_autocall.0.bed russian_autocall.17.bim

russian_autocall.0.bim russian_autocall.17.bnt@

russian_autocall.0.bnt@ russian_autocall.18.bed

russian_autocall.1.bed russian_autocall.18.bim russian_autocall.3.bed

russian_autocall.1.bim russian_autocall.18.bnt@ russian_autocall.3.bim

russian_autocall.1.bnt@ russian_autocall.19.bed russian_autocall.3.bnt@

russian_autocall.10.bed russian_autocall.19.bim russian_autocall.4.bed

russian_autocall.10.bim russian_autocall.19.bnt@ russian_autocall.4.bim

russian_autocall.10.bnt@ russian_autocall.2.bed russian_autocall.4.bnt@

russian_autocall.11.bnt@ russian_autocall.2.bim russian_autocall.5.bed

russian_autocall.12.bed russian_autocall.2.bnt@ russian_autocall.5.bim

russian_autocall.12.bim russian_autocall.20.bed russian_autocall.5.bnt@

russian_autocall.12.bnt@ russian_autocall.20.bim russian_autocall.6.bed

russian_autocall.13.bed russian_autocall.20.bnt@ russian_autocall.6.bim

russian_autocall.13.bim russian_autocall.21.bed russian_autocall.6.bnt@

russian_autocall.13.bnt@ russian_autocall.21.bim russian_autocall.7.bed

russian_autocall.14.bed russian_autocall.21.bnt@ russian_autocall.7.bim

russian_autocall.14.bim russian_autocall.22.bed russian_autocall.7.bnt@

russian_autocall.14.bnt@ russian_autocall.22.bim russian_autocall.8.bed

russian_autocall.15.bed russian_autocall.22.bnt@ russian_autocall.8.bim

russian_autocall.15.bim russian_autocall.23.bed russian_autocall.8.bnt@

russian_autocall.15.bnt@ russian_autocall.23.bim russian_autocall.9.bed

russian_autocall.16.bed russian_autocall.23.bnt@ russian_autocall.9.bim

russian_autocall.16.bim russian_autocall.24.bed russian_autocall.9.bnt@

russian_autocall.16.bnt@ russian_autocall.24.bim russian_autocall.fam

russian_autocall.17.bed russian_autocall.24.bnt@

To create this set of files using Broad intensity data, you will need the following:

• 1. A directory with .gtc.txt files (see above for instructions on how to download)

  2. A SampleSheet linking the chip barcode to the Sample ID. It needs to be comma-separated in the standard Illumina format. This is usually called SampleSheet.csv

  3. The binary PLINK file root path that you want to create the Evoker project for

Once you have the files listed above, run the following command:

makeEvokerInput --indir my_gtc_dir/ --outdir my_evoker_project/ --bfile russ_pchip-qc --outname russian_qc --samplesheet SampleSheet.csv

The following command line options are available:

A lot of temporary files will be created in your scratch directory defined as $sloc in your ricopili.conf file.

To see the scripts progress, check the ${loloc}evoker_info file, where $loloc is the location of your Ricopili log files. Here is example log file output:

/home/ makeEvokerInput --indir ~/gtc_dir --outdir ~/outdir/ --bfile russ_pchip-qc --outname russian_qc --samplesheet russian_sample_sheet.csv convert2int.1993 Tue_Aug_26_15:13:03_2014

/home/ makeEvokerInput --indir ~/gtc_dir --outdir ~/outdir/ --bfile russ_pchip-qc --outname russian_qc --samplesheet russian_sample_sheet.csv paste1.179 Tue_Aug_26_15:38:52_2014

/home/ makeEvokerInput --indir ~/gtc_dir --outdir ~/outdir/ --bfile russ_pchip-qc --outname russian_qc --samplesheet russian_sample_sheet.csv paste2.18 Tue_Aug_26_15:40:31_2014

/home/ makeEvokerInput --indir ~/gtc_dir --outdir ~/outdir/ --bfile russ_pchip-qc --outname russian_qc --samplesheet russian_sample_sheet.csv paste3.2 Tue_Aug_26_15:41:58_2014

/home/ makeEvokerInput --indir ~/gtc_dir --outdir ~/outdir/ --bfile russ_pchip-qc --outname russian_qc --samplesheet russian_sample_sheet.csv paste4.1 Tue_Aug_26_15:44:11_2014

/home/ makeEvokerInput --indir ~/gtc_dir --outdir ~/outdir/ --bfile russ_pchip-qc --outname russian_qc --samplesheet russian_sample_sheet.csv excl.26 Tue_Aug_26_16:08:12_2014

/home/ makeEvokerInput --indir ~/gtc_dir --outdir ~/outdir/ --bfile russ_pchip-qc --outname russian_qc --samplesheet russian_sample_sheet.csv finished Tue_Aug_26_16:20:03_2014

Starting Evoker from the Broad Server

To use Evoker from the Broad server, you will need to have X11 available on your computer!

1. Run the following command to login to the Broad server with X11 enabled:

ssh -X myusername@ni.broadinstitute.org

2. Run the Evoker java script as follows (make sure to include the path if it's not in your rp_bin directory).

java -Xmx1024m -jar Evoker.jar

3. This will open the program on your computer and will look like this:

4. Go to File, and then click on the menu option "Open directory"

5. Enter the directory containing the Evoker input files (--outdir from the makeEvokerInput script). (Note, copy and paste doesn't work…)

6. Enter "OK". If the project loaded successfully, then you will not receive an error message.

7. To test whether everything works, then click on the "Random" button at the bottom left hand of the screen. This will generate a plot for a random SNP that looks like this.

If you successfully generate a random plot, then you can continue with more advanced features of Evoker such as recalling individual points, excluding individuals, and creating an approve/exclude SNP list.

Evoker Documentation

The Evoker documentation is located here.