Peter Dayan (Moderator)
Max Planck Institute for Biological Cybernetics & University of Tübingen (Tübingen, Germany)

Ishita Dasgupta
DeepMind (NY, USA)

Learning from simulations: new tradeoffs for accuracy and cost

Computations (like planning and inference) are expensive. Mental simulations can offload and amortize these costs to offline processes. Given limited storage, it is "ecologically rational" to prioritize computations that are frequent in the environment. I will start with evidence that humans amortize internal computations in a way sensitive to environment query distribution, as well as a model for this behavior. However, simulation can be done in service of a query presented by the environment (e.g. inferring the best path to a fixed goal in a given maze), as well as to generate the queries themselves (e.g. sampling the goal or imagining the maze). I will discuss open questions for the tradeoffs in cost, accuracy and flexibility, for these different kinds of simulation -- both for understanding natural intelligence as well as designing artificial systems.

Christopher Gagne
Max Planck Institute for Biological Cybernetics & University of Tübingen (Tübingen, Germany)

Risk-sensitive planning and sampling in simple and complex generative models

In a world replete with past disappointments and future threats and dangers, an adaptive agent might ponder the causes of its miserable lot and ways of mitigating prospective problems. Pondering the past can be seen as a form of rumination, and pondering the future as a form of worry. Although worry therefore shares many characteristics with planning, and has indeed been described in exactly those terms by both clinical psychologists and those who worry frequently, there have been few attempts to formalize it in the modern language of model-based computation and simulation. In this talk, I will provide a formal framework for characterizing the foundations of adaptive worry, and show applications of this framework at two radically different cognitive scales. Prospective problems typically arise as unlikely, unpleasant, tail, events from distributions of possible outcomes. In our framework, these are conceptualized in terms of a generalized notion of worst-case risk (called CVaR), parametrized by a form of risk aversion. Rational risk-averse agents will plan to reduce worst-case risks as best as possible; we relate such planning in a very simple world model to a form of worry that involves thinking through problematic scenarios and their possible solutions. Given evidence that various forms of planning can occur as unconscious offline replay, we can predict that individual differences in the contents of neural replay will correlate with behavioral measures of risk-aversion. However, the simplicity of the world model precludes investigation of more naturalistic, conscious, worry, which is often verbal in nature. We therefore turn to what are increasingly being seen as the most sophisticated programmatic world models, namely large-scale language systems, and, as an example of contentful worry, show how we can use CVaR to capture the phenomena of catastrophizing, a type of worry characterized by progressively worsening ‘what-if’ chains-of-thought

Quentin Huys
Max Planck UCL Centre for Computational Psychiatry and Ageing Research & Division of Psychiatry and Institute of Neurology UCL (London, UK)

Negative biases in choosing what to think about

Mental illness is frequently characterised by negative alterations in thought preferences. In depression, pessimistic biases lead to negative interpretations. In anxiety disorders, the future is thought to be dangerous. In paranoid states, intentions of others are bad. These thought biases appear to be intimately related to emotional biases. Indeed, a very powerful emotion regulation technique is to alter the contents of thoughts. However, the relationship between thoughts and emotions, and the neurobiological determinants of this mutual relationship remain incompletely understood. In this talk, I will discuss a number of mostly behavioural studies examining this relationship, starting with how affective reactions influence internal decisions about what to think about. These studies identified reactive aversive inhibition as influencing not only external decisions, but also internal decisions about which aspect of a plan to pursue. In order to gain a more detailed algorithmic view on the influence of affective information on thought processes than is possible with behaviour, I will then describe a more recent study using MEG decoding to examine how biases in representations during choice relate to individual differences. Finally, I will close by attempting to integrate these findings within a theoretical proposal assigning emotions a resource-rational role in cognitive processes.

Jutta Joormann
Yale University (New Haven, USA)

Repetitive Negative Thinking in Psychopathology: Examining underlying mechanisms

Repetitive negative thinking – often in the form of worry or rumination- is a transdiagnostic feature of many forms of psychopathology. This feature of psychopathology is very distressing and not yet well understood. In particular, it seems critical to examine cognitive processes that underlie this inability to stop thinking and reorient attention to goal-relevant thoughts and features of the situation. In this talk, repetitive negative thinking and its role in psychopathology is examine more closely as well as the relation to basic cognitive functions such as attention, memory and inhibitory control. Focusing on depression and rumination, underlying cognitive processes as well as specificity to disorders will be discussed.

Rachel L Bedder (Co-organiser)
Princeton University (Princeton, USA)

Rumination as State Inference

Rumination, the act of repeatedly thinking about one’s own negative experience and feelings, is a highly unpleasant experience. Cognitive psychology has suggested a range of reasons for why people exhibit this behavior, including being part of the memory consolidation process of spontaneous thought, and the continued rejection of effective prospective plans. However, no accounts have mathematically formalized these methods in order to explain how various biases held by high trait ruminators (e.g. inability to inhibit negative information, tendency towards abstract thinking) may cause prolonged and repeated episodes of this style of negative thinking. In this talk I will present an integrated set of modelling features based on reinforcement learning which propose that rumination is a process of sampling from memory in order infer the latent state of one’s current experience and identify the optimal policy to resolve one’s negative feelings. I will suggest how biases held by trait ruminators cause them to sample overly negative states from memory causing them to make aberrant judgements about their current circumstances, not allowing them to identify the optimal policy and terminate the ruminative episode.

Peter Hitchcock (Co-organiser)
Brown University, (Providence, USA)

Treating Rumination and Worry as Mental Actions

Rumination and worry are potent risk factors for depression and anxiety disorders. These thinking styles predict poor clinical treatment response, and change little during standard treatments. This has motivated therapies to target them directly, such as Rumination-Focused Cognitive-Behavioral Therapy. Yet, a fundamental limitation to developing such therapies is that we have only a poor basic-science understanding of how mental actions such as rumination and worry are learned (and why they are difficult to learn). To address this, we developed a novel task, The Cognitive Actions Task. Participants must learn the best action in a set of 2-armed bandits, in matched Cognitive and Overt conditions. The task isolates two key challenges of cognitive (i.e., mental) action learning: Cognitive actions require performing a mental operation, and yield no sensory cues to the action undertaken. These challenges are reflected in the Cognitive condition, which requires taking the sum or difference of two numbers, and wherein doing so affords no cues to scaffold a representation of one's action as such. I will present behavioral results that confirm performance is impaired in the Cognitive (vs. Overt) condition, and ongoing computational modeling to capture this impairment precisely. I will argue for the general utility of the approach exemplified by this study for computational psychiatry: Rather than only seeking to understand individual differences (design "fine scalpels"), we can conduct studies with the aim of discovering principles that will improve psychotherapies (forge "strong swords").