This single-author preprint analyzes why a minimal GPT trained on exhaustive 2-digit addition still fails to generalize to 3-digit addition, not as a single accuracy problem but as a staged sequence of computational barriers. Even though the local digit-transition rules are already saturated during 2-digit training, the remaining failure unfolds through four experimentally separable stages: a layout barrier, a carry-flag semantic error at the hundreds position, a conditional recomposition bottleneck, and a late carry-conditioned tens residual.
Key Points
Exhaustive 2-digit addition already covers the local digit-transition rules needed for later addition.
Nevertheless, 3-digit generalization still fails, and the first barrier is a pure layout-shift barrier.
The learned absolute-position model collapses under the 3-digit layout shift, and mixed-layout exposure is the only intervention that materially weakens this first barrier.
After layout repair, the hundreds position behaves like a carry flag rather than a semantic hundreds digit; targeted carry probes reverse the relevant logit margin, whereas a matched extra-data control does not.
After carry repair, the main remaining bottleneck is conditional recomposition, and high-conditioned tail data outperforms a matched control, high-only data, and tail-only data.
The same ordering reappears in a 2-layer, width-32 bridge experiment, where the post-recomposition residual errors become overwhelmingly tens-only.
In a separate 10-seed late-stage study, a sign-aware tens repair raises exact match on the hardest thousands-carry suite from 0.664 to 0.822.
Taken together, the study argues that arithmetic benchmarks should be used not only as single-number contests, but as controlled settings that expose which computational barrier remains active.
Seine A. Shintani. Arithmetic OOD Failure Unfolds in Stages in Minimal GPTs. arXiv (Preprint, Version 1, 2026).
DOI: 10.48550/arXiv.2603.26828
Keywords: arithmetic OOD failure, minimal GPT, length generalization, positional role aliasing, carry semantics, conditional recomposition, out-of-distribution evaluation