Get started with FireCloud

IMPORTANT: This is the legacy GATK documentation. This information is only valid until Dec 31st 2019. For latest documentation and forum click here

created by Tiffany_at_Broad

on 2018-03-28

This guide teaches you how to launch an analysis in FireCloud with your own workflows and data. If you do not have your own workflows or data, you can take advantage of the Featured workspaces to see how pipelines are configured and run a dummy analysis on a small sample file. We have the GATK4 Best Practice pipelines and our $5 genome analysis pipeline featured. Learn how to use Featured Workspaces in this video.

Note: Before you can run the steps here or in the video, you must: * register for a FireCloud account * obtain access to a FireCloud billing project

9 steps to launching your analysis on your data

Each step is followed by a screenshot of the action. Some steps are linked to tutorials where you can read more detail.

Step 1

Create a method in the method repository. FireCloud only accepts workflows written in WDL at this time.

Method Repository | Create New Method

:-------------------------:|:--------------------------:|

|

Step 2

Create a workspace.

Step 3

Now you will edit the workspace to prepare it for your analysis. Upload your data (BAMs, vcfs, etc.) to the Workspace’s Google bucket (video or written instructions). A link to your bucket is in the workspace's Summary tab.

Step 4

Update your workspace's data model to display links to the files you uploaded in Step 3. Create Tab-separated value files (saved as .txt) that organize your metadata by participant, sample, pair, sets of participants, samples, or pairs.

See the attached templates for the import format and required fields for each entity. Once your metadata files are setup, navigate to the Data tab in the workspace, click "Import Metadata", and import the participant file (required), sample file (optional), pair file(optional), participantset file (optional), other set files, etc. in that order. * participantuploadtemplate * sampleuploadtemplate * pairuploadtemplate * participantsetuploadtemplate * samplesetuploadtemplate * pairsetuploadtemplate

Step 5

With the data and metadata setup, you are ready to configure the method in the workspace. Import the method you created in Step 1. * In the workspace navigate to Method Configurations -> Import Configuration->Import from the Method Repository. * Search and find your method. * Click "Select Configuration" and "Use Blank Configuration" because you have not set one up yet. * Select the root entity type (participant, sample, pair, set, etc.) the WDL is meant to run on. * Click "Import Method." * You will notice a message saying that in order to allow other users of this workspace to run this method, you will need to grant them access. Click "Grant Permission" to allow other WRITERS or OWNERs of the workspace to run this method. You can also click "Customize Permissions" and adjust individually.

Import Method | Use Blank Configuration

:-------------------------:|:--------------------------:|

|

Step 6

Configure the method using the WDL inputs json to see the attribute values that you need to fill in. Notice the format for each attribute type in the screenshot below. See this written or this video tutorial for a full list of formatting requirements and to learn how to reference files in the data model and/or workspace attributes in your analysis.

Step 7

Click “Launch Analysis.” Select the entities that you want to analyze. Read more about Defining Expressions if you want to analyze a sample_set or other set.

Step 8

Monitor the job. Refresh the page to see submission, workflow, and task status updates. If the submission status turns red, check the workflow's failures section to see a description of what occurred. The log files are available per workflow and are very helpful for debugging. Read more about workflow statuses and troubleshooting here.

Step 9

Done?! If there are no errors, the workflow(s) will complete and the workspace status will be green. You can take a look at the outputs in the Monitor tab within each workflow or in the Data tab (if the method configuration is set up to write there).

Try these steps out and let us know about your experience. If you find something missing or different than you expected, please let us know in the Comments section.

Attachments:

participant_upload_template.txt

sample_upload_template.txt

pair_upload_template.txt

participant_set_upload_template.txt

sample_set_upload_template.txt

pair_set_upload_template.txt

Updated on 2019-01-25