Tips&Tricks

This bias is likely due to several reasons:

• Differences in read length. This changes the mappability of some genomic regions

• Differences in mapping algorithms (often bwa aln for ancient/historical samples and bwa mem for contemporary samples)

• aDNA damage leading to more singletons in ancient/historical samples

• Differences in sequencing depth and endogenous content

The bias between historical and contemporary samples in the population structure analysis can be reduced/removed by applying the following filters:

• Estimating genotype likelihoods on historical and contemporary samples separately and using the sites that are in common between the two for the angsd run including both historical and contemporary samples (it is usually faster to filter out sites in the final beagle rather than restricting the angsd analysis to these sites)

• CallableLoci analysis. GATK3 contains a tool for this. It identifies regions of the genome with excessive coverage (likely regions incorporated to the nuclear genome from mtDNA and/or cpDNA, gene duplications), low coverage (potentially gene deletions), low mapping quality (likely repetitive regions where mapped reads have multiple best hits) and no coverage, and callable. This analysis is done on bam files.

- Use samples with sequencing depth > 10X to get better statistical power for identifying excessive and low coverage regions.

- In case no samples with high enough sequencing depth are available, combine several samples. I aim to use samples with similar sequencing depth (e.g. 2X each) and downsample to the same sequencing depth if there are large differences (e.g. 1 sample with 5X and one with 2X). I also try to cover all populations represented in the dataset.

 -Run the callable loci analysis on historical and contemporary samples separately and then take regions that are callable in both

- If the dataset contains several species, run the callable loci analysis on all species and use sites that are callable in all species.

• Use mapDamage rescale to reduce aDNA damage

• Use the -trim option in angsd to remove the first and last x bases (e.g., 3) of a read to reduce aDNA damage

• Use a minor allele frequency filter that removes singletons. CAUTION: Be careful with the default maf filter of 0.05 in most tools. This may removes all variants unique to a certain population/species in the dataset depending on sample size!

• Do the callable loci and variable sites filters before filtering for LD.