WGS Annotator (WGSA) - an annotation pipeline for human genome re-sequencing studies

WGSA is an annotation pipeline for human genome re-sequencing studies, to facilitate the functional annotation step of whole genome sequencing (WGS). Currently WGSA supports the annotation of SNVs and indels locally without remote database requests, allowing it to scale up for large WGS studies.

For gene-model based annotation, WGSA integrates the outputs from three annotation tools (ANNOVAR, SnpEff and VEP) for RefSeq and Ensembl GENCODE gene models, plus ANNOVAR outputs using UCSC knowGene model, and provides a summary of variant consequences from the seven annotation results. To further speed up the process for large-scale WGS studies, we have pre-computed annotations for all potential human SNVs (a total of 8,584,031,106 based on human reference hg19 and 8,812,967,043 based on human reference hg38) and use them as a local database (in the resources/precomputed and resources/precomputed_hg38 folder). For SNV-centric resources, WGSA integrated 14 sets of functional prediction scores (CADD, CDTS, FATHMM-MKL, FATHMM-XF, Funseq, Funseq2, RegulomeDB, DANN, fitCons x 4, GenoCanyon, Eigen & Eigen-PC, GenoSkyline-Plus x 127, LINSIGHT, MACIE), 11 conservation scores (bStatistic, GERP++, PhyloP x 5, phastCons x 3, SiPhy), allele frequencies from 5 large-scale re-sequencing studies (1000G, EP6500, ExAC, UK10K, gnomAD), variants in 4 disease related databases (ClinVar, COSMIC, GWAS_catalog, GRASP2), among others (see list of resources). For indel and SV-centric resources, WGSA integrated MetaRNN and StrVCTVRE. For regulatory region-centric resources, WGSA integrated predicted regulatory regions from multiple epigenomics projects (see list of resources). WGSA also contains rich functional annotations for non-synonymous SNVs and genes from our dbNSFP database, including deleteriousness prediction scores from SIFT, SIFT4G, Polyphen2-HDIV, Polyphen2-HVAR, LRT, MutationTaster2, MutationAssessor, FATHMM, MetaSVM, MetaLR, MetaRNN, VEST4, PROVEAN, M-CAP, REVEL, MutPred, MVP, MPC, PrimateAI, GEOGEN2, BayesDel_addAF, BayesDel_noAF, ClinPred, LIST-S2, ALoFT .

WGSA is provided both as an Amazon Machine Image (AMI) ready to run out-of-the-box and a downloadable version. WGSA and its resources are freely available for academic usage. Licenses are required for non-academic usage of some of the resources, such as ANNOVAR, CADD, CDTS, GenoCanyon, GenoSkyline-Plus, LINSIGHT, SpliceAI and VEST4, Polyphen2, ClinPred and REVEL (in dbNSFP). WGSA does not grant the non-academic usage of those resources, so please contact the original content provider for that purpose.

You can sign up for our email list for future release announcements.

Please report any bug you found or any suggestions or comments to xmliu.uth{at}gmail.com or xiaomingliu{at}usf.edu.

Upcoming:

The next version will focus on updating gene models.

Current Version:

WGSA v0.95 AMI

WGSA pipeline AMI: WGSAv095 (ami-030460dca5b954dda in AWS US East region - N. Virginia)
A guide for using WGSA (example of using WGSA via Amazon Web Service) can be found here.
Guidance for using external resources (COSMIC, SPIDEX, CADD indel) can be found here.

WGSA v0.95 downloadable version

The downloadable version (total size of ~1.5 Tb) is available:
- Use command wget -c http://database.liulab.science/files/WGSA095.tar
- Or zip and download folders separately from BOX.
- Due to the big size of resources, I strongly recommend using AMI instead of using the downloadable version.
A guide for using the downloadable version can be found here.
- A guide for installing WGSA on an external hard drive attached to a Linux machine can be found here.
Guidance for using external resources (COSMIC, SPIDEX, CADD indel, dbNSFP) can be found here.

Utilities for WGSA

A collection of utilities programs for using WGSA resources and post-processing WGSA annotations can be found here. (update 20190223)

Archives:

Archives of older versions of WGSA can be found here.

Citation:

Liu X*, White S, Peng B, Johnson AD, Brody JA, Li AH, Huang Z, Carroll A, Wei P, Gibbs R, Klein RJ and Boerwinkle E. (2016) WGSA: an annotation pipeline for human genome sequencing studies. Journal of Medical Genetics 53:111-112. [PDF] [preprint] *corresponding author

Changelog:

Update (May 20, 2022): WGSA 0.95 released. This is a minor update focused on the update of resources. Major changes include

StrVCTVRE SV (DEL and DUP) pathogenicity prediction score was added (only supports hg38 and the input file must be in vcf format).
MetaRNN indel pathogenicity prediction score was added.
MACIE pathogenicity prediction score for coding and noncoding SNVs was added.
usDSM pathogenicity prediction score for synonymous SNV was added.
Two PhyloP conservation scores based on 241 mammals and 36 mammals (no human) added.
CDTS function score based on human genome diversity was added.
Genotypes of Chagyrskaya Neandertal were added.
dbNSFP was updated to 4.3c (also support 4.3a).
clinvar was updated to 20220425.
GWAS catalog was updated to e105_r2022-04-07.
dbSNP was updated to b155.
Ensembl Regulatory Build was updated to release104 (hg19) and release106 (hg38).
Default human reference version is now hg38 (use -v hg19 for annotating hg19 based variants).

Update (Feb 8, 2021): WGSA 0.9 released. This is a major update focused on the update of gene-model based annotations. Major changes include

VEP is updated to v100.
ANNOVAR is updated to 20200608 version.
All precomputed gene-model based annotations are updated accordingly.
gnomAD genome is updated to v3.1 (only support hg38). Allele frequencies of controls, non-Cancer, non-Neuro, non-TOPMed subsets added.
gnomAD exome is updated to r2.1.1 (only support hg38). Allele frequencies of controls, non-Cancer, non-Neuro, non-TOPMed subsets added.
dbSNP is updated to b154.
dbNSFP is updated to 4.1c (also support 4.1a).
clinvar is updated to 20210131.
GWAS catalog is updated to e100_r2021-01-29.

Update (June 11, 2020): WGSA 0.85 released. This is a minor update focused on the update of resources. Major changes include

SpliceAI is added.
gnomAD genome updated to r3.0 (only support hg38).
CADD updated to v1.6.
GTEx updated to v8 (only support hg38).
dbNSFP updated to 4.0c (also support 4.0a).
GWAS catalog updated to e100_r2020-06-04.
clinvar updated to 20200609.
Ensembl_Regulatory_Build updated to release 100.
Support vep 96 and later version on the fly (tested on vep 96).

Update (April 19, 2019): WGSA 0.8 released. This is a major update focused on the update of gene-model based annotations. Major changes include

VEP is updated to 94.
SnpEff updated to v4.3t.
All precomputed gene-model based annotations are updated accordingly.
Indel-annotation-via-SNV-annotations is presented in a way that pseudo-SNV annotations have one-to-one correspondence. Please check Indel annotations via SNV annotations.
dbNSFP is updated to 4.0b2.
Clinvar is updated to 20190311.
GWAS catalog is updated to gwas_catalog_v1.0.2-associations_e93_r2019-01-31.
GTEx and Geuvadis eQTLs' targets now use gene symbols instead of Ensembl IDs.

Update (December 12, 2018): WGSA v0.76 released. This is a minor update focused on the update of annotation resources. Please see the guidance before downloading. Major changes include

The default dbNSFP in WGSA is now dbNSFP4.0b1c. WGSA also supports dbNSFP4.0b1a with dbNSFPa_variant option for academic usage of additional deleteriousness prediction scores including Polyphen2, VEST4 and REVEL. See this guidance for using dbNSFP4.0b1a.
Aloft is no longer an independent resource but provided via dbNSFP.
The default ANNOVAR program for indel annotation is now version 20180416, which supports Ensembl gene model for hg38. HGVSp presentation for indel is now supported.
As fathmm-XF coding and noncoding scores are comparable, the two scores are now combined into one fathmm-XF score with additional information for its origin (coding or noncoding).
CADD is updated to v1.4.
WGSA's CADD indel support now assumes the results are from CADD v1.4, which now support both hg19 and hg38.
gnomAD is updated to 2.1. The number of alt allele homozygotes and allele frequencies of controls subsets are now included.
clinvar is updated to 20180930.
dbSNP is updated to b151.
QTEx eQTL is updated v7.
phyloP placental conservation score for hg38 is updated to 30way.
phastCons placental conservation score for hg38 is updated to 30way.
Added phyloP primate 17way conservation score for hg38.
Added phastCons primate 17way conservation score for hg38.
Added bStatistic for hg38.
GWAS catalog is updated to e93.
Known miRNA database miRdb is updated to 22.
miRNA target database TargetScan is updated to v7.2.

Update (July 2, 2018): WGSA v0.75 released. This is a minor update focused on the addition of annotation resources. Please see the guidance before downloading. Major changes include

While preparing input file, duplicated variants will be shown on screen but no longer removed from the input file.
Added FATHMM-XF score, a whole genome deleteriousness prediction score.
Added predicted regulatory elements (15-state and 25-state models) for 127 cell types from the Roadmap epigenomes.
Added GeneHancer, predicted target genes for enhancers (and promoters).
Added eQTLs from the Geuvadis project.
Added dbNSFP v3.4 collection of deleteriousness prediction scores for missense SNVs
Added Aloft, a deleteriousness prediction score for stop-gain SNVs.
Added LINSIGHT, a whole genome function prediction score.
Added bStatistic, a measure of background selection and conservation based on comparative genomics.

Update (February 15, 2018): WGSA v0.71 released. This is a minor update for supporting gnomAD r2.0.2 and the ANNOVAR version of spidex. Changes include

gnomAD updated to r2.02. One column added to the annotation indicating whether the variant is within low complexity region or segment duplication region.
The third-party spidex resource file changed to hg19_spidex.txt. Users can download this file from the ANNOVAR website.

Update (August 6, 2017): WGSA v0.7 released. This is a major update focused on hg38 support as well as resource updates. A clean re-download is recommended. Please see the guidance before downloading. Major changes include

1. WGSA07 add options to specify whether the input file format is vcf or tsv and whether the variant coordinates are in hg19 or hg38. The full usage is java WGSA07 [setting_file] <-m maximum_number_of_GB_memory_to_use> <-t maximum_number_of_threads_to_use> <-v hg19_or_hg38> <-i vcf_or_tsv>

Add support for coordinates of variants in hg38. For annotating variants with coordinates in hg38, annotation resources native in hg38 will be used if applicable, otherwise, the hg38 coordinates will be converted to hg19 coordinates then be annotated with those native in hg19.
Added allele frequencies of the Genome Aggregation Database (gnomAD)
Added GenoSkyline-Plus scores, a tissue-specific deleteriousness prediction score for 127 cell types.
Added topologically associated domains (TADs)
Added Vindijia Neanderthal genotypes. Genotypes of Altai Neanderthal and Denisova updated.
Ensembl Regulatory Build updated to Ensembl release 88
dbSNP updated to b150
GWAS catalog updated to e88_r2017-05-29
clinvar updated to 20170530
GTEx updated to v6p
Eigen and EigenPC updated to v1.1
ORegAnno updated to 2015.12.22
funseq2 updated to 2.1.6
dbNSFP updated to 2.9.3
GenoCanyon updated to 1.0.2

Update (Sept. 21, 2016): WGSA v0.65 released. This update focused on annotation resources. Major changes include

Eigen and EigenPC scores (Nat. Genet. 48, 214–220) added.
GenoCanyon score (Sci. Rep. 5, 10576) added.6
FANTOM5 enhancer target genes, promoter robust set (phase 1+2), enhancer expression (phase 1), enhancer robust set added.
Super Enhancer (Cell 155, 934–947) added.
Genome Mappability Score (GMS) added.
Duke mappability scores averaged over 300bp windows added.
dbSNP updated to build 147.
ClinVar updated to 20160802.
dbNSFP updated to v2.9.1.
FANTOM5 enhancer updated to permissive set (phase 1+2).
Support CADD indel annotation output. See the guidance here.

The following folders under resources need to be updated if you have the downloaded version of WGSA06 on your computer:

The following folders have been added: Eigen, EigenPC, GenoCanyon, GMS, SuperEnhancer.
The files under the following folders have been modified: clinvar, dbNSFP, dbSNP, Duke_Mapability, FANTOM5, javaclass.

A clean update, i.e. deleting all files under those folders (if exist) and re-download files from the updated resources, is recommended.

Update (Mar. 3, 2016): WGSA v0.6 released. This update focused on gene model annotations. Major changes include

ANNOVAR annotation results were updated as to its Dec. 2015 version. ANNOVAR program was updated to the Feb. 2016 version, which fixed the multiple-thread bug of its Dec. 2015 version.
snpEff annotation results and the program were updated as to its version 4.2. The new 'ANN' format annotations were used.
VEP annotation results and the program were updated as to its version 83. The results of the LoF plugin by LOFTEE are now included.
Precomputed ANNOVAR annotation results for all SNVs with UCSC knowGene model is added.
Transcript-specific annotations are now included by default in the precomputed ANNOVAR/snpEff/VEP annotation results.
Tool by gene model results now can be selected separately. That is, user can choose whether to include ANNOVAR/ensembl, ANNOVAR/refseq, ANNOVAR/ucsc, snpEff/ensembl, snpEff/refseq, VEP/ensembl, VEP/refseq results separately. Accordingly, the format of configuration file has changed. Please refer to the new format here.
User can choose whether to get gene model based annotations (i.e. ANNOVAR/snpEff/VEP x ensembl/refseq/ucsc) of SNVs using precomputed results (recommended for large data sets) or using the annotation tools on-the-fly (faster for small data sets).
User can choose a working directory to store the intermediate files. All intermediate files are gzipped files by default to save disk space.
Gzipped input files are now supported.
Resources of GTEx eQTLs, Roadmap epigenomics peak calls (narrowPeaks for >1000 epigenomics data sets), and allele frequencies of the ExAC r0.3 nonTCGA and nonpsych subsets are added.
User need to download their own copy of COSMIC resource for annotation due to licence requirements. Please see the guidance here.

The following folders under resources need to be updated if you have the downloaded version of WGSA055 on your computer:

The contents under the following folders have been deleted: COSMIC, IntegratedSNV, FAMTOM5 .
The following folders have been added: GTEx, precomputed, Roadmap_peaks, FANTOM5.
The files under the following folders have been modified: 1000Gmask, clinvar, dbSNP, ENCODE, EnhancerFinder, Ensembl_regulatory_build, ESP6500, ExACr0.3, GRASP, GWAS_catalog, hg19, human_ancestor_GRCh37_e71, javaclass, ORegAnno, repeatmasker, scSNV, snoRNA_miRNA.

A clean update, i.e. deleting all files under those folders (if exist) and re-download files from the updated resources, is recommended.

Update (Oct. 6, 2015): WGSA v0.55 released. This update focused on annotation resources. Major changes include

DANN, fitCons and EnhancerFinder resources added;
Support annotation using the SPIDEX free non-commercial version but independent license/download needed.
Genome-wide ranks added to CADD, DANN, FATHMM-MKL, fitCons, funseq-2, GERP++, phastCons, phyloP, SiPhy;
CADD, dbSNP, snoRNA/miRNA, miRNA targets, clinvar updated;
Multiple alt alleles of indels in dbSNP, ExAC, ESP6500 and 1000Gp3 have been separated and left-normalized;
Annotation results using standard variant list file as input will retain all columns of the input file;
Allows options to turn off integrated annotations of ANNOVAR/SnpEff/VEP x Refseq/Ensembl;
Bugs fixed with repeat mask and 1000g mask annotation;
In case there are multiple rows in dbNSFP match the variant, those rows are combined to a single row.

The following folders under resources need to be updated if you have the downloaded version of WGSA05 on your computer: 1000Gp3, CADDv1.3, DANN, ESP6500, EnhancerFinder, ExACr0.3, GERP, PhyloP, SiPhy, clinvar, dbSNP, fathmmMKL, fitConsv1.01, funseq2, javaclass, phastCons, snoRNA_miRNA. A clean update, i.e. deleting all files under those folders (if exist) and re-download files from the updated resources, is recommended.