Synaptic basis for reinforcement learning
Many psychiatric disorders involve monoamines such as dopamine and serotonin, synapse-related genes, and brain regions such as the frontal and limbic systems. All of these are relevant to learning for flexibly adapting to the environment. We thus clarified how transient changes in dopamine signaling regulate synaptic plasticity and learning in the nucleus accumbens (NAc), a region receiving strong projections of dopaminergic neurons,
The NAc consists the neuron groups expressing the dopamine D1 receptor (D1 cells) and the other half of cell groups expressing the D2 receptors (D2 cells). When an unexpected reward is given during the conditioned learning by the reward, the dopamine concentration transiently increases from the baseline level. It is well known that this signal causes classical conditioning that associates conditional stimuli (CS) such as sounds with rewards such as water (unconditional stimuli, US). We monitored and manipulated dopamine activity in vivo by optogenetics and fiber photometry, and manipulated glutaminate and dopamine by 2 photon uncaging of caged-glutamate and optogenetics in NAc slices. We found that transient increase in dopamine concentration enhanced the plasticity of dendritic spines in D1 cells within a narrow time window, a synaptic eligibility trace (Yagishita et al., Science, 2014), and regulated conditional learning (Yamaguchi,…, Yagishita, BioRxiv). We unexpectedly noticed that the D1-dependent conditioning broadly generalized to similar sensory stimuli that have never been learned (generalized learning). If this over-prediction was incorrect, dopamine showed transient decrease, which induced spine plasticity of D2 cells for refining the prediction (discrimination learning) (Iino,…, Yagishita, Nature, 2020). These data provided a novel foundation for understanding reinforcement learning from the synaptic level.
Currently, we are expanding our research to serotonin and prefrontal cortex to understand learned behaviors from synaptic functions.