NBI26: Neural Basis of Intention

Goals

In 2026 the Auditory and Neuroscience group will address unsolved scientific questions related to TWO topics:

1. The first is intention where we will direct our knowledge towards novel brain-computer interfaces. We want to understand how the brain responds to internal intent and external stimuli and then build mathematical models to describe the results. We seek in these projects to understand how sensory signals are represented, how meaningful information is encoded in the neural responses, how they guide perception and behavior; and how we can close the loop from decoding these signals to control devices or inform actions. We also seek to use the measurements to answer scientific questions about how the brain works, as well as investigate real-time brain-computer interfaces. 

2. The second topic we shall pursue involves the idea of utilizing travelling waves (TWs) and recurrent NN to implement the temporal coherence (TC) algorithm, a principle of neuronal operations that we have invoked for over 10 years to explain how the brain is able segregate complex mixtures of sounds (multiple speakers or musical instruments). A colleague of ours here at UMD, Behtash Babadi, has formulated a beautiful theory for how TWs might actually be able to implement the TC and that combines any mixtures of sensory stimuli (not just acoustics). We will design experiments in Telluride to measure and confirm ideas that combine TWs and TC, a very interesting and challenging goal that will unite the interest of our AUD team with the Neural Dynamics topic  team.

Projects

All our experiments will involve actual EEG recordings, but also this year combined with  fNIRs and pupilometry. Here are some examples below in more detail.

• Decoding Cognitive Intentions The goal is to decode the intentions to pursue one of many  tasks, such as directing attention between different sources in a mixture of 2 or more sources. We will conduct these experiments with speech and with musical signals as well as tone sequences of different parameters. We shall compare the results to those from an AI LCR model that is formulated by Nima Mesgarani.

• Decoding Motor Intentions We seek to decode the intended motor action upon receiving a cue (i.e., to move arm right or left, or to lift one of 5 fingers.). This will build upon the experiments we started last year. 

• Using linear and nonlinear decoding methods to disentangle motor and auditory signals in EEG recordings, and to assess the superposition of these signals when estimated from EEG recordings of  purely motor (miming) and purely perceptual (listening) conditions. 

• We will explore parallels between intention dynamics and reasoning dynamics in LLMs, specifically how LLMs represent and reason about intent, expectations and goal-directed behavior.

• Saliency and foundational models for low-SNR domains, such as brain science. We will seek to replicate and build upon Elhilali/Slaney work on saliency of sources in a mixture.

• Finalize the benchmark to compare and quantify the performance of 4 different EEG systems (Wet, Dry, Band, in-ear).

All of our work will be oriented towards real-time demos.  Telluride’s BCI work will be our test case, across multiple types of hardware. The BCI work (and demos) has been a great vehicle to collaborate with other groups in Telluride.

Materials and Equipment:

We will bring several types of hardware to Telluride, including the normal 64 channel wet-electrode recording system from BrainVision, and mobile dry electrode systems, as well as behind the ear cee-grid and fNIRS. 

Talks and Tutorials

In order to ensure successful engagement of Telluride participants in the activities of the group, we anticipate a number of tutorials that will benefit the workshop, including:

• Basic auditory processing, for cochlea to cortex

A tutorial on basic auditory processing from the cochlea to the cortex will provide a comprehensive overview of how sound is transformed into neural signals and processed by the brain. Beginning with the cochlea, the tutorial would explain its role as the auditory system's mechanical transducer, converting sound waves into electrical signals through the movement of hair cells. The tutorial would also cover relay stations including the brainstem, discussing models of signal timing and intensity coding. Finally, we will explore the auditory cortex and discuss modeling of temporal and spectral modulations.

• Basic decoding: linear to DNNs

Common methods used to explore brain representations in the auditory system are temporal transfer functions which provide a mathematical description of how auditory neurons respond to changes in sound intensity or frequency over time. These functions are crucial for understanding auditory perception because they can model the way different parts of an auditory signal, such as the onset, offset, and sustained phases of a sound. By examining these temporal dynamics, researchers can infer how sounds are represented in the brain, and how these representations contribute to our perception of sound patterns and specifically speech. A newer body of work exploring deep learning offers a different perspective, particularly in terms of adaptively adjusting parameters during training, allowing them to model intricate dependencies and variations in auditory signals that linear models might miss. With all their computational prowess, these methods are not showing great improvements. The tutorial will explore the general framework for decoding methods.

• Differentiable neural modeling

This tutorial explores the emerging field of bio-physically plausible, differentiable neural modeling, a hybrid approach combining precise mathematical modeling with adaptability of deep learning. Differentiable neural models integrate traditional differential equations into neural network architectures, allowing these networks to not only learn from data but also to incorporate established scientific principles. The tutorial will discuss foundations of differentiable modeling including automatic differentiation, vectorization, and just-in-time compilation; as well as tools commonly used for differentiable computing including JAX.

Recommended reading

Keywords: Audio perception, brain decoding, speech and music perception, real-time, EEG, BCI, Machine Learning