Abstract: When faced with novel situations, people are able to marshal relevant considerations from a wide range of background knowledge and put these to use in inferences and predictions. What permits us to draw in globally relevant information and reason over it coherently? Here, we explore the hypothesis that people use a combination of distributed and symbolic representations to construct bespoke mental models tailored to novel situations. We propose a computational implementation of this idea -- a ``Model Synthesis Architecture'' (MSA) -- using language models to implement global relevance-based retrieval and model synthesis and probabilistic programs to implement bespoke, coherent world models. We evaluate our MSA as a model of human judgments on a novel reasoning dataset. The dataset -- built around a `Model Olympics` domain of sports vignettes -- tests models' capacity for human-like, open-ended reasoning by requiring (i) judgments about novel causal structures described in language; (ii) drawing on large bodies of background knowledge; and (iii) doing both in light of observations that introduce arbitrary novel variables. Our MSA approach captures human judgments better than language model-only baselines, under both direct and chain-of-thought generations from the LM that supports model synthesis. These results suggest that MSAs can be implemented in a way that mirrors people's ability to deliver locally coherent reasoning over globally relevant variables, offering a path to understanding and replicating human reasoning in open-ended domains.
Abstract: Seeing is fast and thinking is slow. This difference is often taken to justify one of the most significant divisions between parts of the mind, the border between perception and cognition. It is often used as a premise in arguments, in both philosophy and psychology, about the perceptual or cognitive nature of other mental processes. But why is seeing fast and thinking slow? This paper starts from the observation that the speed difference is puzzling – flatfooted computational analyses suggest that, if anything, the speed difference should go the other way (thinking should be fast, and seeing slow), in part because the problems perception solves appear to be vastly larger than those cognition solves. It then develops a novel computational explanation, based on asymmetries in ‘computational amortization,’ or the way in which typical cases of perception, but not cognition, are suited to exploit repeated structure in the problems they solve. This difference turns out to be deep – explaining several other contrasts between perception and cognition – and illuminating recent findings in computational neuroscience and human-AI comparisons.
Abstract: Perception solves computationally demanding problems at lightning fast speed. It recovers sophisticated representations of the world from degraded inputs, often in a matter of milliseconds. Any theory of perception must be able to explain how this is possible; in other words, it must be able to explain perception's computational tractability. One of the few attempts to move toward such an explanation has been the information encapsulation hypothesis, which posits that perception can be fast because it keeps computational costs low by forgoing access to information stored in cognition. I argue that we have no compelling reason to believe that encapsulation explains (or even contributes to an explanation of) perceptual tractability, and much reason to doubt it. This is because there exist much deeper computational challenges for perception than information access, and these threaten to make the costs of access irrelevant. If this is right, it undermines a core computational motivation for encapsulation and sends us back to the drawing board for explanations of perceptual tractability.