IJCAI 2024 Tutorial in Jeju, Korea

Brain Encoding and Decoding with Deep Neural Networks: 

Methods, Applications and Future Directions

Abstract

In recent years, Deep Learning (DL) has made significant progress, with large foundational models showing near-human performance in various tasks across different areas. These advances highlight DL’s potential to help us understand how the human brain processes and interprets these types of information, particularly through brain encoding and decoding. Brain encoding involves mapping stimuli to predict the neural responses they trigger, while brain decoding reconstructs perceived or imagined stimuli from brain activity. DL models, which excel at identifying and manipulating features in data, are well-suited for linking stimuli with brain activities and vice versa. This tutorial will focus on how DL can support brain encoding and decoding, with a particular emphasis on language, visual and speech perception. We will explain the principles and methods of using DL for brain encoding and decoding, presenting both overview of well cited related works and examples of specific studies. We will also discuss the challenges and future directions in this field. By featuring insights from experienced researchers in each area, we aim to provide a thorough and informative overview of the intersection between DL and cognitive neuroscience, encouraging future research, welcoming both junior scholars and experts in this rapidly evolving area.