TARE

TARE: Exact Incremental Re-annotation Across Reference Database Releases

Joshua Terranova, Jiapeng Zhang

University of Southern California

USC Viterbi School of Engineering

Thomas Lord Department of Computer Science

Abstract

Biologists annotate genomes by searching their proteins against reference databases such as Pfam, which publish a new release every few weeks to months. Each release forces a full re-search, although most of the database has not changed: 86.6% of Pfam’s entries matched between releases 38.1 and 38.2 after excluding build metadata and comments. Reusing the old results is hard, because the search programs are fixed binaries whose printed statistics depend on the size of the whole database. We present TARE (Tool Annotation Reuse, Exact), a wrapper that reruns an unmodified program only on the entries that can matter, with the program’s own database-size setting pinned to the new release, so that the program itself prints the numbers a full run would print. TARE then tests the replay by adding back entries it left out, and refuses, returning an ordinary full run, if any line changes. Under the contract, every output line except comment headers is byte-identical to a full rerun; the contract assumptions are tested on each update, not proved. In the bounded timing reported here, TARE also reruns the program in full on 299 sampled records drawn from those with at least one replay candidate, so a hidden dependency that affects at least 1% of that candidate-bearing population ships with probability at most 5%, assuming a valid audit oracle; records with an empty candidate set are covered by the contract conditions (pinned Z, per-family --cut_ga thresholds), not by the audit sample.

On Pfam updates, TARE is 4.24× faster than a full rerun with a lightweight sampled audit (median of nine genomes on a dedicated server; 95% CI 4.17–4.68), and 3.49× faster under the historical bounded audit that carries the candidate-population guarantee above (same nine genomes; 95% CI 3.32–4.10). Across 715,704 output lines from HMMER and Infernal, not one differed from a full run. TARE refused DIAMOND, whose first filter reads bytes of the neighboring database entry, and BLAST+, whose unpinned probe rows fit none of TARE’s column rules.

Illustration of TARE

First, when a new database release arrives, TARE finds the entries that can matter and pins the program’s database-size setting to the new release, so the unmodified program reruns only on those entries and prints the numbers a full run would, while every other result is reused (left). Then, TARE tests the replay by adding back a sample of the entries it left out and rerunning them in full, and compares every output line against the replay. If every line matches, the replay is kept; if any line changes, TARE refuses and returns an ordinary full run (right).

Selective replay of entries that can matter

We rerun the unmodified program only on the entries that can change the output, and reuse every other result.

Pinned database size for exact numbers

We pin the program's own database-size setting to the new release, so the program itself prints the numbers a full run would.

Audit of the replay, or refusal

We add back a sample of the entries we left out and compare every output line, and if any line changes we return an ordinary full run.

Experiments on Real Database Releases

E-value scales with database size. Z grows from 27,481 to 30,134, and the same hit’s E goes from 8.5×10⁻⁵ to 9.3×10⁻⁵.

Pinning Z restores the full-run E-value. A subset rerun prints 1.2×10⁻⁵ without the pin and 9.3×10⁻⁵ with -Z 30134.

Score cutoffs do not depend on Z. Z grows while the hit and no-hit points stay put.

Audit size from the miss-rate bound. The curve (0.99)ⁿ falls to 5% at n = 299.

Avoiding Silent Errors

We also show that TARE can avoid silent errors, producing exactly what a full run would print where a naive update quietly returns wrong numbers.

Updated with TARE

We can update an annotation without a full rerun and still get every output line exactly as a full run prints it.

Updated with naive reuse

Even though it reruns exactly the right families, the output differs from a full run on 9,083 of 9,094 lines for one genome.

Comparison to Prior Work

We compare our approach to prior methods for updating an annotation after a database release, without being able to modify the annotation program. Specifically, we consider general-purpose caches that rerun everything when the reference changes, and approximate incremental search.

Comparison of TARE with prior methods. Left: first-run cost relative to a stock run (500 proteins, Pfam-A; Riker was measured on a laptop). Middle: update speedup over a full rerun against the fraction of the reference that changed (HMMER, server); caches rerun everything on a new release, while TARE follows its cost model, with measured medians shown with 95% intervals. Right: share of output lines identical to a full run for naive reuse of cached lines, an approximate incremental tool (iSeqSearch), and TARE, which is exact or refuses.

Chance that a wrong update ships against the number of affected records, for six kinds of injected coupling. The pink line and shaded region are the measured random-pad results over 50 trials with their Wilson 95% interval, and the dashed line is the random-pad formula. The other curves are the audit bound for audit sizes of 100, 299 and 1,000, and the dotted line marks the 5% allowed miss rate.

Ablations

In order to evaluate the individual effects of each technique in TARE, we do an ablation study on HMMER updates of Pfam, with injected-coupling trials for the safety checks.

Ablation of TARE’s techniques. (a) Share of output lines that differ from a full run, without and with the pin. (b) Wall-clock time of the first design (replay every protein against all candidates, check with four full-size replays) and the shipped design (batched replay, sampled check); black ticks mark a stock run. (c) Share of wrong updates that shipped on injected hidden or distant coupling, for each check design. (d) Cost of the audit as a share of a stock run.

BibTex

@misc{terranova2026pinreplayrefuseconditional,

title={Pin and Replay, or Refuse: Conditional Exactness and Probabilistic Validation Across Reference Releases},

author={Joshua Terranova and Jiapeng Zhang},

year={2026},

eprint={2611.02182},

archivePrefix={arXiv},

primaryClass={q-bio.GN},

url={https://arxiv.org/abs/2611.02182},

}

Page updated

Report abuse