Multi-Class Anomaly Detection

Multi-Class Anomaly Detection with Differential Dual-Channel Reconstruction (DDR)

Introduction

A practical inspection line rarely produces just one kind of product. Ideally, a single model should cover every category on the line, rather than one model per product. This is the multi-class unsupervised setting, and it is much harder than it sounds: a single network must hold many different "normal" appearances at once.

The dominant recipe today freezes a large pre-trained Vision Transformer as the encoder and trains only a light bottleneck and decoder to reconstruct its normal features. Anything the decoder fails to reconstruct is flagged as a defect. Two problems sit quietly inside this recipe. First, the features taken from different encoder layers, and from different product categories, live at wildly different scales, so the reconstruction target itself is unbalanced before training even begins. Second, the decoder can cheat: it learns to simply copy its input through, an identity shortcut that reconstructs defects just as well as normal regions and therefore hides them.

Framework

DDR is built on a frozen DINOv3 encoder and adds three components that work together:

Context-aware Layer Calibrator (CLC) builds a stable reconstruction target by recentering features against the class token, normalizing each layer independently, and then fusing the layers. This brings every layer back onto a comparable scale and removes the feature-norm imbalance.

Differential Linear Attention (DLA) is the decoder's spatial mixer. Each attention head is split into a zero-diagonal global branch, which aggregates context from other positions only, and a diagonal self branch, which carries a position's own response. The two are recombined through a token-wise dynamic coefficient and stabilized with RMSNorm, so the model can control how much of "itself" each token is allowed to copy.

Dual-view Stochastic Reconstruction (DSR) runs the shared bottleneck and decoder twice with independent dropout, both passes supervised by the same calibrated target. This regularizes the decoder implicitly, without synthetic defects and without any explicit pairwise loss.

Results

Evaluated on MVTec-AD, VisA, and Real-IAD, DDR performs consistently well, with the clearest margins on the large-scale Real-IAD benchmark, where it surpasses the Dinomaly baseline on every image-level and pixel-level metric. Ablations show a two-stage pattern: CLC delivers the dominant stabilization gain, while DLA and DSR contribute a complementary gain that only appears once the target has been calibrated.

The same model transfers to medical imaging without modification. On the BMAD benchmark it reaches 93.92 image-level AUROC on brain MRI and 93.94 on retinal OCT, ahead of the memory-bank method PatchCore (91.65 and 91.55). Throughout, DDR stays simple, purely reconstruction-based, and entirely free of synthetic-anomaly supervision.

Page updated

Google Sites

Report abuse