The Mila Neural-AI Reading Group
Talks on cutting-edge research at the intersection of Neuroscience and AI
Attend for free, in-person or online on Fridays at 3pm ET!
Serotonin predictively encodes value
Serotonin predictively encodes value
December 01, 2023 at 3:00 pm
December 01, 2023 at 3:00 pm
Mila Room F01 and Google Meet
Mila Room F01 and Google Meet
Abstract
Abstract
The in vivo responses of dorsal raphe nucleus (DRN) serotonin neurons to emotionally-salient stimuli are a puzzle. Existing theories centred on reward, surprise, salience, and uncertainty individually account for some aspects of serotonergic activity but not others. Here we find a unifying perspective in a predictive code for value, a quantity that combines biological constraints with the representation of future reward used in reinforcement learning. Through simulations of trace conditioning experiments common in the serotonin literature, we show that our theory, called value prediction, explains phasic activation of serotonin neurons by both rewards and punishments, preference for surprising rewards but absence of a corresponding preference for punishments, and contextual modulation of tonic firing–observations that currently form the basis of many and varied serotonergic theories. To empirically test our theory, we analyzed tetrode recordings of identified serotonin neurons in mice undergoing trace conditioning, finding single-neuron and population-level activity patterns well within 0.1 Hz / neuron of our predictions; a surprisingly close match. Finally, we directly compared value prediction against quantitative formulations of existing ideas and found that our theory best explains both within-trial activity dynamics and trial-to-trial modulations, almost always by a large margin. Overall, our results show that previous models are not wrong, but incomplete, and that reward, surprise, salience, and uncertainty are simply different faces of a predictively encoded value signal. By unifying previous theories, our work represents an important step towards understanding the potentially heterogeneous computational roles of serotonin in learning, behaviour, and beyond. An interactive version of the value prediction model can be found online at https://efharkin.com/blog/2023-10-value-widget/
The in vivo responses of dorsal raphe nucleus (DRN) serotonin neurons to emotionally-salient stimuli are a puzzle. Existing theories centred on reward, surprise, salience, and uncertainty individually account for some aspects of serotonergic activity but not others. Here we find a unifying perspective in a predictive code for value, a quantity that combines biological constraints with the representation of future reward used in reinforcement learning. Through simulations of trace conditioning experiments common in the serotonin literature, we show that our theory, called value prediction, explains phasic activation of serotonin neurons by both rewards and punishments, preference for surprising rewards but absence of a corresponding preference for punishments, and contextual modulation of tonic firing–observations that currently form the basis of many and varied serotonergic theories. To empirically test our theory, we analyzed tetrode recordings of identified serotonin neurons in mice undergoing trace conditioning, finding single-neuron and population-level activity patterns well within 0.1 Hz / neuron of our predictions; a surprisingly close match. Finally, we directly compared value prediction against quantitative formulations of existing ideas and found that our theory best explains both within-trial activity dynamics and trial-to-trial modulations, almost always by a large margin. Overall, our results show that previous models are not wrong, but incomplete, and that reward, surprise, salience, and uncertainty are simply different faces of a predictively encoded value signal. By unifying previous theories, our work represents an important step towards understanding the potentially heterogeneous computational roles of serotonin in learning, behaviour, and beyond. An interactive version of the value prediction model can be found online at https://efharkin.com/blog/2023-10-value-widget/
Speaker
Speaker
Emerson Harkin is a doctoral candidate in computational neuroscience at the University of Ottawa with Richard Naud. Before that, he trained as an electrophysiologist during his M.Sc. with Jean-Claude Béïque and as a molecular biologist during his B.Sc. with Paul Albert (also at the University of Ottawa). His computational work takes a “top-down meets bottom-up” approach influenced by his background in physiology, linking the biophysical features of individual neurons to the normative functions of neural systems, with a particular focus on understanding how the physiology of the dorsal raphe nucleus shapes the role of the serotonin system in reinforcement learning.
Emerson Harkin is a doctoral candidate in computational neuroscience at the University of Ottawa with Richard Naud. Before that, he trained as an electrophysiologist during his M.Sc. with Jean-Claude Béïque and as a molecular biologist during his B.Sc. with Paul Albert (also at the University of Ottawa). His computational work takes a “top-down meets bottom-up” approach influenced by his background in physiology, linking the biophysical features of individual neurons to the normative functions of neural systems, with a particular focus on understanding how the physiology of the dorsal raphe nucleus shapes the role of the serotonin system in reinforcement learning.
The goal of this reading group is to bring researchers interested in both Neuroscience and AI together.
The goal of this reading group is to bring researchers interested in both Neuroscience and AI together.
To learn more, check out our previous talks. If you are interested in presenting or suggesting work to be discussed at the reading group, please reach out to us at neuralai.montreal@gmail.com.
To learn more, check out our previous talks. If you are interested in presenting or suggesting work to be discussed at the reading group, please reach out to us at neuralai.montreal@gmail.com.
Presentations do not need to be based on published work. They may describe new, previously published, or in-progress research. We welcome presentations on anything Neuro-AI – theory, methods, applications and more!
Featured Past Talks
Featured Past Talks
December 5, 2019
Speakers: Blake Richards, Irina Rish, Gaël Varoquaux, Yoshua Bengio
October 5, 2022
Speaker: Philip Sabes
NeurIPS 2023 Competition
25 August, 2023
Speaker: Guillaume Durandau
Reach out to us at neuralai.montreal@gmail.com if you have any questions!© 2023 The Mila Neural-AI Reading Group. The cover image was generated using Stable Diffusion.