Abstract Why do some of the world's most powerful AI systems still struggle to reason about cause and effect? Causal discovery, the ability to look at a system and ask what drives what, sits at the heart of scientific thinking. Yet, despite their remarkable fluency, Large Language Models (LLMs) appear to take shortcuts, leaning on pattern recognition where genuine causal logic is required. We know that LLMs stumble here. What we don't know is exactly where, why, and how badly. This paper maps that territory. We put state-of-the-art LLMs through a systematic stress test across six dimensions that any real-world causal task might throw at them: the size of the causal graph, the richness and quality of variable descriptions, the resolution of the underlying data, and the sophistication of how questions are posed. What we found is striking. Something as simple as the order in which variables are listed or the presence of seemingly harmless extra context can send even the most capable models into confident, catastrophic errors. The result is a practical guide to LLMs' blind spots in causal reasoning: a clear-eyed picture of when these tools are genuinely useful in a causal discovery pipeline and when trusting them could quietly lead you astray.
Slides
Photos
Participants: 20