Code over Words:

Overcoming Semantic Inertia via Code-Grounded Reasoning

Manjie Xu1,2,5,6* Isabella Yin3* Xinyi Tu4 Chi Zhang1,5✉️ Yixin Zhu2,1,5,6,7✉️
* Equal Contribution, ✉️ Correspondence Author

1 Institute for AI, Peking University 2 School of Psychological and Cognitive Sciences, Peking University 3 Tsinghua International School 4 University of California, Berkeley 5 State Key Lab of General AI, Peking University 6 Beijing Key Laboratory of Behavior and Mental Health, Peking University 7 Embodied Intelligence Lab, PKU-Wuhan Institute for Artificial Intelligence

Intro

Large Language Models (LLMs) struggle with Semantic Inertia: the inability to inhibit pretrained priors (e.g., “Lava is Dangerous”) when dynamic, in-context rules contradict them. We probe this phenomenon using Baba Is You, where physical laws are mutable text rules, enabling precise evaluation of models’ ability to override learned priors when rules change. We quantatively observe that larger models can exhibit inverse scaling: they perform worse than smaller models when natural language reasoning requires suppressing pre-trained associations (e.g., accepting “Lava is Safe”).

Our analysis attributes this to natural language encoding, which entangles descriptive semantics and logical rules, leading to persistent hallucinations of familiar physics despite explicit contradictory rules. Here we show that representing dynamics as executable code, rather than descriptive text, reverses this trend and enables effective prior inhibition.

An intuitive example to our setting: -->

GoTo Arxiv Paper

Quantative Experiments of Semantic Inertia

Before introducing our LCV pipeline, we establish its empirical motivation through a probing analysis of a critical question: Does scaling model size automatically resolve semantic inertia?

We evaluate three model families (Llama-3, Pythia, Qwen2.5) spanning 160M to 70B parameters. Negative values indicate semantic inertia—defaulting to learned associations over contextual logic.

In Natural Language prompting (red lines), larger models often exhibit worse performance due to semantic interference, with models showing clear inverse scaling. Code Grounding (green lines) decouples logical operations from semantic priors, restoring positive scaling laws where computational scale translates to improved contextual reasoning.

Code Grounded Vistas

More Demos

"Code as a Scaffold, Logic as a Solution."
LCV exhibits systematic generalization: because it operates on an intermediate code representation, it can compose rules it has never seen together, provided the underlying syntax (Python Code) remains consistent.

@inproceedings{xu2026code,

title={Code over Words: Overcoming Semantic Inertia via Code-Grounded Reasoning},

author={Xu, Manjie and Yin, Isabella and Tu, Xinyi and Zhang, Chi and Zhu, Yixin},

booktitle={ACL 2026 Findings},

year={2026}

}