Schmutzi at 10: Still Standing

A decade-long benchmark of ancient DNA mitochondrial contamination estimators

The Problem: Every Ancient DNA Sample is Contaminated

Ancient DNA is rarely pure. No matter how carefully a bone fragment is excavated, cleaned, and processed, modern human DNA will find its way into the extract. Laboratory workers, museum curators, archaeologists who handled the specimen centuries ago, all leave microscopic traces. In a typical ancient DNA library, anywhere from a few percent to nearly all the mitochondrial reads may come from modern contaminants rather than the individual who once owned the bone.

Getting the contamination estimate wrong has real consequences. Overestimate contamination and you reject a perfectly usable specimen; underestimate it and you draw phylogenetic or demographic conclusions from a mixture of ancient and modern sequences. Accurate contamination estimation is therefore one of the most important quality-control steps in ancient DNA research.

Two tools have shaped how the field approaches this problem on mitochondrial genomes: ContamMix, introduced by Fu et al. in 2013 as part of a landmark study on the human evolutionary timescale, and Schmutzi, which I published in 2015. Ten years on, I wanted to see how both hold up on a rigorous set of simulated mixtures. The results vindicate Schmutzi but also reveal exactly why ContamMix can work (if you cheat a bit).

How the Two Methods Work

ContamMix (Fu et al. 2013)

ContamMix frames contamination estimation as a mixture problem. Given a panel of 311 diverse modern human mitochondrial genomes plus a consensus sequence of the sample’s own endogenous genome, it uses Bayesian inference (implemented as an MCMC) to estimate what fraction of the sequenced reads is best explained by the endogenous consensus versus any one of the 311 reference haplotypes.

The critical input is that endogenous consensus. In practice, researchers typically build it by mapping to the reference mitochondrial genome, calling a consensus from the reads, and feeding that consensus back to ContamMix. If the sample is already heavily contaminated, however, that consensus is itself a mixture of endogenous and contaminant sequences—which contaminates the contamination estimate. ContamMix has no mechanism to break this circularity on its own. Thank you to Abigail Ramsoee at Copenhagen University who sent me the script that they use internally.

Schmutzi (Renaud et al. 2015)

Schmutzi addresses the circularity with an iterative, joint estimation strategy. Ancient DNA carries a characteristic signature of post-mortem damage: cytosine deamination produces C→T mismatches preferentially at the 5′ end of reads and G→A mismatches at the 3′ end. Modern contaminating reads, freshly extracted from living (or recently deceased) cells, show minimal or no such damage.

Schmutzi exploits this damage asymmetry to classify reads as likely-ancient or likely-modern, estimate an initial contamination rate, call an endogenous consensus from the putatively-ancient reads, and then refine the contamination estimate using that consensus—iterating until convergence. It can be run in two modes:

• nopred: uses the allele frequency database alone to discriminate endogenous from contaminating reads, without attempting to build a consensus of the endogenous genome. Contamination is estimated purely from how reads align relative to population-level allele frequencies.
• wpred: iteratively predicts the endogenous (contaminant-free) genome and incorporates that predicted consensus into the database. By adding a sample-specific endogenous reference alongside the population allele frequencies, the model can use both damage patterns and sequence divergence from the predicted endogenous genome to separate endogenous reads from contaminants.

The key difference from ContamMix is that Schmutzi does not require a pre-existing accurate endogenous consensus supplied by the user. In wpred mode it builds one internally from first principles using the damage signal, breaking the circularity that complicates ContamMix analyses.

Experimental Design

To benchmark both methods fairly, I simulated contamination in silico for five samples:

For each sample, I mixed the endogenous BAM with a single modern human contaminant genome (haplogroup U5a2a, accession HM229344.1) at 21 mixing rates ranging from 1% to 99% contamination, and then down-sampled to seven coverage levels: 0.5×, 1×, 2×, 4×, 8×, 16×, and 32×. That yields over 700 test cases per tool and mode—more than 2,000 benchmarked combinations in total.

The choice of contaminant matters because both tools ultimately rely on sequence differences between the endogenous and contaminating genomes to count the fraction of reads that came from each source. The more divergent the endogenous genome is from the contaminant, the more informative sites are available to discriminate the two, and the easier contamination is to quantify. The five endogenous samples span a wide range of divergence from U5a2a: the two archaic hominins (Neanderthal, Denisovan) are maximally divergent—hundreds of fixed differences separate them from any modern human—while the three modern human haplogroups (D4a1d, L2, T2a1) are conspecific with the contaminant, leaving only haplogroup-level polymorphisms as discriminating sites. This gradient from easy (archaic) to hard (same-species modern human) is the central stress-test of the benchmark. The targets in order of decreasing difficulty are T2a1, D4a1d, L2, chagyrskaya8 and denisova21.

For ContamMix I ran it in two variants:

• Standard: feeding it the consensus called from the raw mixed BAM—the typical real-world workflow shown in sections 1 and 4.
• Hacked (oracle): feeding it the true endogenous genome sequence—an intentionally unrealistic upper-bound condition included in the supplemental panels to show ContamMix’s performance ceiling.

In every plot below, the dashed diagonal is the ideal y = x line: a perfect estimator would place all points exactly on it. Error bars show 95% credible intervals. An ✕ marks cases where the tool produced no usable estimate.

Results

1. ContamMix in Standard Use

In standard practice, ContamMix takes a consensus sequence built from the mixed BAM itself and uses it as the endogenous reference. When contamination is low that consensus is predominantly ancient and the estimator tracks the true rate accurately. As contamination rises the consensus absorbs more and more contaminant sequence, ContamMix then estimates contamination relative to an increasingly chimeric reference, and the estimates become unreliable. We illustrate with haplogroup L2, a modern human sample where endogenous and contaminant are conspecific.

Key takeaway: Standard ContamMix struggles when the contaminant and endogenous are close but for archaic humans, it is reliable at low contamination, where the consensus built from the reads is still predominantly endogenous. As contamination rises the consensus becomes chimeric and the estimate degrades. Section 4 shows what ContamMix can achieve when this circularity is removed.

2. Schmutzi Without Prediction (nopred): Database Alone

In nopred mode, Schmutzi estimates contamination using the allele frequency database without building any endogenous consensus. For the L2 modern human sample, damage patterns are the primary discriminating signal—the allele frequency database alone must separate endogenous from contaminating reads when both are human.

The bimodal pattern makes intuitive sense. At low contamination most reads carry ancient damage and the allele frequency database correctly identifies a small contaminant fraction. As contamination increases the mixture of damaged and undamaged reads creates an ambiguous landscape, and a phase transition occurs when modern reads sufficiently dilute the damage signal. More coverage helps because the damage-pattern statistics become more reliable with more reads.

3. Schmutzi With Prediction (wpred): Endogenous Consensus Incorporated into the Database

The wpred mode iteratively predicts the endogenous genome and adds it to the database alongside the population allele frequencies. For the L2 modern human sample, the predicted consensus adds limited extra discriminatory power relative to nopred—endogenous and contaminating genomes are conspecific, so sequence divergence cannot assist the separation as it would for an archaic hominin.

Comparing nopred (section 2) and wpred (section 3) for L2 shows that adding the predicted consensus provides a modest improvement in the moderate-contamination range at 16× and 32×.

4. ContamMix with the Oracle Genome: the Performance Ceiling

To ask how well ContamMix can perform when the circularity problem is removed, we ran it with the true endogenous genome supplied directly—the oracle (hacked) variant. This is not a realistic workflow; it shows what the Bayesian model achieves given a perfect input. The same L2 sample at 8×, 16×, and 32×:

Key takeaway: ContamMix’s Bayesian model is fundamentally sound: given a clean endogenous reference it tracks contamination accurately across the full 0–100% range at all coverages tested. The limitation in standard use is not the model but the input—the chimeric consensus that arises from the reads themselves as contamination rises.

5. Additional Samples (click to expand)

The panels below cover the remaining four samples—the two archaic hominins (Chagyrskaya Neanderthal, Denisovan) and three modern human haplogroups (same-species contamination: D4a1d, L2, T2a1)—each at 8×, 16×, and 32× coverage for all three estimation approaches.

Chagyrskaya8 (Neanderthal) — archaic + modern human contaminant

ContamMix (standard)

Schmutzi nopred

Schmutzi wpred

Denisova21 (Denisovan) — archaic + modern human contaminant

ContamMix (standard)

Schmutzi nopred

Schmutzi wpred

D4a1d (modern human, East Asia) — same-species contamination

ContamMix (standard)

Schmutzi nopred

Schmutzi wpred

L2 (modern human, Africa) — same-species contamination

ContamMix (standard)

Schmutzi nopred

Schmutzi wpred

T2a1 (modern human, West Eurasia) — same-species contamination

ContamMix (standard)

Schmutzi nopred

Schmutzi wpred

Discussion

ContamMix: a Sound Model, a Fragile Input

Sections 1 and 4 together make a clean separation: ContamMix’s Bayesian haplotype, mixture model is excellent, section 4 shows it tracking contamination across the full 0–100% range when handed a clean endogenous reference. The problem is that in standard practice (section 1) that reference is built from the mixed reads themselves. As contamination rises the consensus becomes a chimera, and the estimate degrades with it. The model is not broken; the input is.

Schmutzi was designed precisely to break that circularity by using ancient DNA damage as a model-free signal to bootstrap the endogenous consensus from scratch.

When Is Schmutzi Better?

In any realistic scenario where you do not have external knowledge of the endogenous genome, Schmutzi is still the appropriate choice. Its contamination estimates at low contamination levels (the region most relevant for quality filtering: “is this sample <5% contaminated?”) are accurate at about 32× coverage across all five sample types tested here.

When Does ContamMix Add Value?

If you have an external source of clean endogenous sequence e.g. a second library from the same individual, a published genome of a closely related specimen, or a low-contamination fraction separated by chemistry then ContamMix with that reference breaks the circularity and becomes a powerful tool (as the oracle panels in the supplemental section illustrate).

Conclusion

Summary in two sentences: ContamMix could work well—the oracle experiment shows its Bayesian model is accurate across the full contamination range when given a clean endogenous reference. In standard use, however, that reference is built from the contaminated reads themselves, and the estimate degrades as contamination rises; Schmutzi sidesteps this by bootstrapping the endogenous consensus from the ancient damage signal, correctly identifying low-contamination specimens—the cases that matter most even at modest depth a decade after its publication.

All plots, scripts, and data files are available in the project repository.

References

Fu Q, Mittnik A, Johnson PLF, Bos K, Lari M, Bollongino R, Sun C, Giemsch L, Schmitz R, Burger J, Ronchitelli AM, Martini F, Cremonesi RG, Svoboda J, Bauer P, Caramelli D, Castellano S, Reich D, Pääbo S, and Krause J. (2013). A revised timescale for human evolution based on ancient mitochondrial genomes. Curr Biol, 23:553–9. [ContamMix]

Renaud G, Slon V, Duggan AT, Kelso J. (2015). Schmutzi: estimation of contamination and endogenous mitochondrial consensus calling for ancient DNA. Genome Biol 16, 224. https://doi.org/10.1186/s13059-015-0776-0

Previous post
Website release