Where Neuroscience Meets AI

The brain remains the only known example of a truly general-purpose intelligent system. The study of human and animal cognition has revealed key insights, such as the ideas of parallel distributed processing, biological vision, and learning from reward signals, that have heavily influenced the design of artificial learning systems. Many AI researchers continue to look to neuroscience as a source of inspiration and insight. A key difficulty is that neuroscience is a vast and heterogeneous area of study, encompassing a bewildering array of subfields. In this tutorial, we will seek to provide both a broad overview of neuroscience as a whole, as well as a focused look at two areas -- computational cognitive neuroscience and the neuroscience of learning in circuits -- that we believe are particularly relevant for AI researchers today. We will conclude by highlighting several ongoing lines of work that seek to import insights from these areas of neuroscience into AI, and vice versa.

NeurIPS virtual site

Schedule

Video

1. Introduction / background (15 min) - Recorded - Slides

2. Cognitive neuroscience (30 min) - Recorded - Slides

3. Q/A (10 min) - Live

4. Learning circuits and mechanistic neuroscience (30 min) - Recorded - Slides

5. Q/A (10 min) - Live

6. Recent advancements at the intersection (25 min) - Recorded - Slides

7. General discussion (30 min), sli.do questions - Live

Ask Questions here!

https://app.sli.do/event/92gy6nuo

References

Section 1 - Cognitive Neuroscience

Textbooks

• Gazzaniga, M., Ivry, R. B., & Mangun, G. R. (2018). Cognitive Neuroscience. W.W. Norton & Company.
• O’Reilly, R. C., & Munakata, Y. (2000). Computational Explorations in Cognitive Neuroscience. MIT Press.
• Abbot, L. F., & Dayan, P. (2001). Theoretical neuroscience. MIT Press.

Reviews: Innateness

• Zador, A. M. (2019). A critique of pure learning and what artificial neural networks can learn from animal brains. Nature Communications.
• Pinker, S. (2003). The Language Instinct: How the Mind Creates Language. Penguin UK.
• Hinton, G. E., & Nowlan, S. J. (1987). How learning can guide evolution. Complex Systems

Reviews: Vision

• Goodale, M. A., & Milner, A. D. (1992). Separate visual pathways for perception and action. Trends in Neurosciences,
• Andersen, R. A., & Buneo, C. A. (2002). Intentional Maps in Posterior Parietal Cortex. Annual Review of Neuroscience .
• Whitwell, R. L., Milner, A. D., & Goodale, M. A. (2014). The Two Visual Systems Hypothesis: New Challenges and Insights from Visual form Agnosic Patient DF. Frontiers in Neurology.
• DiCarlo, J. J., & Cox, D. D. (2007). Untangling invariant object recognition. Trends in Cognitive Sciences.
• Dehaene, S., & Cohen, L. (2011). The unique role of the visual word form area in reading. Trends in Cognitive Sciences.
• Kanwisher, N., & Yovel, G. (2006). The fusiform face area: a cortical region specialized for the perception of faces. Phil. Trans. Royal Society of London: B.

Reviews: Memory

• Squire, L. R. (2009). The legacy of patient HM for neuroscience. Neuron.
• McClelland, J. L., McNaughton, B. L., & O’Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological Review.
• O’Reilly, R. C., & Norman, K. A. (2002). Hippocampal and neocortical contributions to memory: advances in the complementary learning systems framework. Trends in Cognitive Sciences.

Reviews: Planning

• Unterrainer, J. M., & Owen, A. M. (2006). Planning and problem solving: from neuropsychology to functional neuroimaging. Journal of Physiology.
• Dolan, R. J., & Dayan, P. (2013). Goals and habits in the brain. Neuron
• Miller, K. J., & Venditto, S. J. C. (2021). Multi-step planning in the brain. Current Opinion in Behavioral Sciences

Other works cited

• Akrami, A., Kopec, C. D., Diamond, M. E., & Brody, C. D. (2018). Posterior parietal cortex represents sensory history and mediates its effects on behaviour. Nature.
• Andersen, R. A., Andersen, K. N., Hwang, E. J., & Hauschild, M. (2014). Optic ataxia: from Balint’s syndrome to the parietal reach region. Neuron.
• Balaguer, J., Spiers, H., Hassabis, D., & Summerfield, C. (2016). Neural Mechanisms of Hierarchical Planning in a Virtual Subway Network. Neuron.
• Chomsky, N. (1976). Reflections on Language. Fontana Press.
• Cohen, N. J., & Squire, L. R. (1980). Preserved learning and retention of pattern-analyzing skill in amnesia: dissociation of knowing how and knowing that. Science.
• Dagher, A., Owen, A. M., Boecker, H., & Brooks, D. J. (1999). Mapping the network for planning: a correlational PET activation study with the Tower of London task. Brain.
• Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron.
• Doll, B. B., Duncan, K. D., Simon, D. A., Shohamy, D., & Daw, N. D. (2015). Model-based choices involve prospective neural activity. Nature Neuroscience.
• Downing, P. E., Jiang, Y., Shuman, M., & Kanwisher, N. (2001). A cortical area selective for visual processing of the human body. Science.
• Epstein, R., & Kanwisher, N. (1998). A cortical representation of the local visual environment. Nature.
• Fernando, C., Sygnowski, J., Osindero, S., Wang, J., Schaul, T., Teplyashin, D., Sprechmann, P., Pritzel, A., & Rusu, A. (2018). Meta-learning by the Baldwin effect. Proceedings of the Genetic and Evolutionary Computation Conference.
• Gabrieli, J. D., Corkin, S., Mickel, S. F., & Growdon, J. H. (1993). Intact acquisition and long-term retention of mirror-tracing skill in Alzheimer’s disease and in global amnesia. Behavioral Neuroscience.
• Goodale, M. A., Milner, A. D., Jakobson, L. S., & Carey, D. P. (1991). A neurological dissociation between perceiving objects and grasping them. Nature.
• Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., Badia, A. P., Hermann, K. M., Zwols, Y., Ostrovski, G., Cain, A., King, H., Summerfield, C., Blunsom, P., Kavukcuoglu, K., & Hassabis, D. (2016). Hybrid computing using a neural network with dynamic external memory. Nature.
• Guo, Z. V., Li, N., Huber, D., Ophir, E., Gutnisky, D., Ting, J. T., Feng, G., & Svoboda, K. (2014). Flow of cortical activity underlying a tactile decision in mice. Neuron.
• Holmes, G. (1918). Dsturbances of visual orientation. British Journal of Ophthalmology
• Hopcraft, R., Holdo, R., Mwangomo, E., Mduma, S., Thirgood, S., Borner, M., Fryxell, J., Olff, J., & Sinclair, A. (2015). Why are wildebeest the most abundant herbivore in the Serengeti ecosystem? In A. Sinclair, K. Metzger, S. Mduma, & J. Fryxell (Eds.), Serengeti IV: Sustaining biodiversity in a coupled human-natural system. University of Chicago Press.
• James, T. W., Culham, J., Humphrey, G. K., Milner, A. D., & Goodale, M. A. (2003). Ventral occipital lesions impair object recognition but not object-directed grasping: an fMRI study. Brain
• Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: a module in human extrastriate cortex specialized for face perception. J. Neuroscience
• Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM.
• Kurth-Nelson, Z., Economides, M., Dolan, R. J., & Dayan, P. (2016). Fast Sequences of Non-spatial State Representations in Humans. Neuron.
• Miller, G. A., Galanter, E., & Pribram, K. H. (1960). Plans and the structure of behavior.
• Miller, K. J., Botvinick, M. M., & Brody, C. D. (2017). Dorsal hippocampus contributes to model-based planning. Nature Neuroscience.
• Milner, A. D., Perrett, D. I., Johnston, R. S., Benson, P. J., Jordan, T. R., Heeley, D. W., Bettucci, D., Mortara, F., Mutani, R., & Terazzi, E. (1991). Perception and action in “visual form agnosia.” Brain.
• Milner, B., Corkin, S., & Teuber, H.-L. (1968). Further analysis of the hippocampal amnesic syndrome: 14-year follow-up study of H.M. Neuropsychologia.
• Murata, A., Gallese, V., Luppino, G., Kaseda, M., & Sakata, H. (2000). Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. J. Neurophysiology.
• Naselaris, T., Bassett, D. S., Fletcher, A. K., Kording, K., Kriegeskorte, N., Nienborg, H., Poldrack, R. A., Shohamy, D., & Kay, K. (2018). Cognitive Computational Neuroscience: A New Conference for an Emerging Discipline. Trends in Cognitive Sciences.
• Neisser, U. (2014). Cognitive Psychology
• Norman, K. A., & O’Reilly, R. C. (2003). Modeling hippocampal and neocortical contributions to recognition memory: a complementary-learning-systems approach. Psychological Review.
• Pflugshaupt, T., Gutbrod, K., Wurtz, P., von Wartburg, R., Nyffeler, T., de Haan, B., Karnath, H.-O., & Mueri, R. M. (2009). About the role of visual field defects in pure alexia. Brain.
• Pritzel, A., Uria, B., Srinivasan, S., Puigdomènech, A., Vinyals, O., Hassabis, D., Wierstra, D., & Blundell, C. (2017). Neural Episodic Control. arXiv [cs.LG]
• Russell, S., & Norvig, P. (2002). Artificial Intelligence: A Modern Approach
• Schapiro, A. C., Turk-Browne, N. B., Botvinick, M. M., & Norman, K. A. (2017). Complementary learning systems within the hippocampus: a neural network modelling approach to reconciling episodic memory with statistical learning. Phil. Trans. Royal Society of London: B.
• Shallice, T. (1982). Specific impairments of planning. Phil. Trans. Royal Society of London: B.
• Shen, E. H., Overly, C. C., & Jones, A. R. (2012). The Allen Human Brain Atlas: comprehensive gene expression mapping of the human brain. Trends in Neurosciences
• Snyder, L. H., Batista, A. P., & Andersen, R. A. (1997). Coding of intention in the posterior parietal cortex. Nature.
• Starck, J. M., Ricklefs, R. E., & Others. (1998). Avian growth and development: evolution within the altricial-precocial spectrum. Oxford University Press.
• Steinmetz, N. A., Zatka-Haas, P., Carandini, M., & Harris, K. D. (2019). Distributed coding of choice, action, and engagement across the mouse brain. Nature.
• Stringer, C., Pachitariu, M., Steinmetz, N., Carandini, M., & Harris, K. D. (2019). High-dimensional geometry of population responses in visual cortex. Nature.
• Szwed, M., Dehaene, S., Kleinschmidt, A., Eger, E., Valabrègue, R., Amadon, A., & Cohen, L. (2011). Specialization for written words over objects in the visual cortex. NeuroImage.
• Tsao, D. Y., Freiwald, W. A., Tootell, R. B. H., & Livingstone, M. S. (2006). A cortical region consisting entirely of face-selective cells. Science.
• van Opheusden, B., Galbiati, G., Z. Bnaya Li, Y., & Ma, W. J. (2017). Modeling Decision Tree Search in a Two-Player Game. Proceedings of the 39th Annual Meeting of the Cognitive Science Society.
• Vesia, M., & Crawford, J. D. (2012). Specialization of reach function in human posterior parietal cortex. Experimental Brain Research.
• Vikbladh, O., Meager, M. R., King, J., Blackmon, K., Devinsky, O., Shohamy, D., Burgess, N., & Daw, N. D. (2018). Two Sides of the Same Coin: The Hippocampus as a Common Neural Substrate for Model-Based Planning and Spatial Memory. Neuron
• Wada, Y., & Yamamoto, T. (2001). Selective impairment of facial recognition due to a haematoma restricted to the right fusiform and lateral occipital region. Journal of Neurology, Neurosurgery, and Psychiatry.
• Wallace, D. J., Greenberg, D. S., Sawinski, J., Rulla, S., Notaro, G., & Kerr, J. N. D. (2013). Rats maintain an overhead binocular field at the expense of constant fusion. Nature.
• Williams, S. C. P., & Deisseroth, K. (2013). Optogenetics. PNAS.
• Wimmer, R. D., Ian Schmitt, L., Davidson, T. J., Nakajima, M., Deisseroth, K., & Halassa, M. M. (2015). Thalamic control of sensory selection in divided attention. Nature.
• Zoccolan, D., Kouh, M., Poggio, T., & DiCarlo, J. J. (2007). Trade-off between object selectivity and tolerance in monkey inferotemporal cortex. J Neuroscience
• Zoccolan, D., Oertelt, N., DiCarlo, J. J., & Cox, D. D. (2009). A rodent model for the study of invariant visual object recognition. PNAS

Section 2 - Circuits and Mechanistic Neuroscience

• Shepherd, Gordon M. The synaptic organization of the brain. Oxford university press, 2004.
• Fee, Michale S., and Jesse H. Goldberg. A hypothesis for basal ganglia-dependent reinforcement learning in the songbird. Neuroscience 198 (2011): 152-170.
• Kornfeld, J., Januszewski, M., Schubert, P., Jain, V., Denk, W., & Fee, M. S. (2020). An anatomical substrate of credit assignment in reinforcement learning. BioRxiv.
• Murdoch, D., Chen, R. and Goldberg, J.H., 2018. Place preference and vocal learning rely on distinct reinforcers in songbirds. Scientific reports, 8(1), pp.1-9.
• Gielow, M.R. and Zaborszky, L., 2017. The input-output relationship of the cholinergic basal forebrain. Cell reports, 18(7), pp.1817-1830.
• Chubykin, A.A., Roach, E.B., Bear, M.F. and Shuler, M.G.H., 2013. A cholinergic mechanism for reward timing within primary visual cortex. Neuron, 77(4), pp.723-735.
• O'Reilly, R.C., Hazy, T.E., Mollick, J., Mackie, P. and Herd, S., 2014. Goal-driven cognition in the brain: a computational framework. arXiv preprint arXiv:1404.7591.
• O'Reilly, R.C., Wyatte, D. and Rohrlich, J., 2014. Learning through time in the thalamocortical loops. arXiv preprint arXiv:1407.3432.
• Guerguiev, J., Lillicrap, T.P. and Richards, B.A., 2017. Towards deep learning with segregated dendrites. ELife, 6, p.e22901.
• Sacramento, J., Costa, R.P., Bengio, Y. and Senn, W., 2018. Dendritic cortical microcircuits approximate the backpropagation algorithm. In Advances in neural information processing systems (pp. 8721-8732).
• Körding, K.P. and König, P., 2001. Supervised and unsupervised learning with two sites of synaptic integration. Journal of computational neuroscience, 11(3), pp.207-215.
• Whittington, J.C. and Bogacz, R., 2019. Theories of error back-propagation in the brain. Trends in cognitive sciences, 23(3), pp.235-250.
• Clancy, K.B., Koralek, A.C., Costa, R.M., Feldman, D.E. and Carmena, J.M., 2014. Volitional modulation of optically recorded calcium signals during neuroprosthetic learning. Nature neuroscience, 17(6), pp.807-809.
• George, D., Lazaro-Gredilla, M., Lehrach, W., Dedieu, A. and Zhou, G., 2020. A detailed mathematical theory of thalamic and cortical microcircuits based on inference in a generative vision model. bioRxiv.
• Heeger, D.J., 2017. Theory of cortical function. Proceedings of the National Academy of Sciences, 114(8), pp.1773-1782.
• Rolls, E.T., 1996. A theory of hippocampal function in memory. Hippocampus, 6(6), pp.601-620.
• Pfeiffer, B.E. and Foster, D.J., 2015. Autoassociative dynamics in the generation of sequences of hippocampal place cells. Science, 349(6244), pp.180-183.
• Carrillo-Reid, L., Yang, W., Bando, Y., Peterka, D.S. and Yuste, R., 2016. Imprinting and recalling cortical ensembles. Science, 353(6300), pp.691-694.
• Müller, M.G., Papadimitriou, C.H., Maass, W. and Legenstein, R., 2020. A model for structured information representation in neural networks of the brain. Eneuro, 7(3).
• Abbott, L.F., Bock, D.D., Callaway, E.M., Denk, W., Dulac, C., Fairhall, A.L., Fiete, I., Harris, K.M., Helmstaedter, M., Jain, V. and Kasthuri, N., 2020. The Mind of a Mouse. Cell, 182(6), pp.1372-1376.
• Turner, N.L., Macrina, T., Bae, J.A., Yang, R., Wilson, A.M., Schneider-Mizell, C., Lee, K., Lu, R., Wu, J., Bodor, A.L. and Bleckert, A.A., 2020. Multiscale and multimodal reconstruction of cortical structure and function. bioRxiv.
• George D, Lehrach W, Kansky K, Lázaro-Gredilla M, Laan C, Marthi B, Lou X, Meng Z, Liu Y, Wang H, Lavin A. A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs. Science. 2017 Dec 8;358(6368).

Section 3 - Recent advancements at the intersection

• Yamins, D. L., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014). Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, 111(23), 8619-8624.
• Yamins, D. L., & DiCarlo, J. J. (2016). Using goal-driven deep learning models to understand sensory cortex. Nature neuroscience, 19(3), 356-365.
• Mante, V., Sussillo, D., Shenoy, K. V., & Newsome, W. T. (2013). Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature, 503(7474), 78-84.
• Song, H. F., Yang, G. R., & Wang, X. J. (2017). Reward-based training of recurrent neural networks for cognitive and value-based tasks. Elife, 6, e21492.
• Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T., & Wang, X. J. (2019). Task representations in neural networks trained to perform many cognitive tasks. Nature neuroscience, 22(2), 297-306.
• Dezfouli, A., Morris, R., Ramos, F. T., Dayan, P., & Balleine, B. (2018). Integrated accounts of behavioral and neuroimaging data using flexible recurrent neural network models. In Advances in Neural Information Processing Systems (pp. 4228-4237).
• Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., ... & Botvinick, M. (2018). Prefrontal cortex as a meta-reinforcement learning system. Nature Neuroscience, 21(6), 860-868.
• Dabney, W., Kurth-Nelson, Z., Uchida, N., Starkweather, C. K., Hassabis, D., Munos, R., & Botvinick, M. (2020). A distributional code for value in dopamine-based reinforcement learning. Nature, 577(7792), 671-675.
• Akrout, M., Wilson, C., Humphreys, P., Lillicrap, T., & Tweed, D. B. (2019). Deep learning without weight transport. In Advances in neural information processing systems (pp. 976-984).
• Miconi, T. (2017). Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks. Elife, 6, e20899.
• Bellec, G., Salaj, D., Subramoney, A., Legenstein, R., & Maass, W. (2018). Long short-term memory and learning-to-learn in networks of spiking neurons. In Advances in Neural Information Processing Systems (pp. 787-797).
• Merel, J., Aldarondo, D., Marshall, J., Tassa, Y., Wayne, G., & Ölveczky, B. (2019). Deep neuroethology of a virtual rodent. In International Conference on Learning Representations.
• Greydanus, S., Koul, A., Dodge, J., & Fern, A. (2018, July). Visualizing and understanding atari agents. In International Conference on Machine Learning (pp. 1792-1801). PMLR.
• Barrett, D. G., Morcos, A. S., & Macke, J. H. (2019). Analyzing biological and artificial neural networks: challenges with opportunities for synergy?. Current opinion in neurobiology, 55, 55-64.
• Morcos, A. S., Barrett, D. G., Rabinowitz, N. C., & Botvinick, M. (2018). On the importance of single directions for generalization. In International Conference on Learning Representations.
• Raghu, M., Gilmer, J., Yosinski, J., & Sohl-Dickstein, J. (2017). Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. In Advances in Neural Information Processing Systems (pp. 6076-6085).
• Puri, N., Verma, S., Gupta, P., Kayastha, D., Deshmukh, S., Krishnamurthy, B., & Singh, S. (2019, September). Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attribution. In International Conference on Learning Representations.
• Sussillo, D., & Barak, O. (2013). Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks. Neural computation, 25(3), 626-649.
• Maheswaranathan, N., Williams, A., Golub, M., Ganguli, S., & Sussillo, D. (2019). Universality and individuality in neural dynamics across large populations of recurrent networks. In Advances in neural information processing systems (pp. 15629-15641).