Biologically Plausible Learning

Program:

15:00-15:05 Welcome

15:05-15:30 invited talk Guillaume Bellec

15:30-15:55 invited talk Raoul Memmesheimer

15:55-16:40 coffee break / poster session

16:40-17:05 invited talk Bojian Yin

17:05-17:30: invited talk Johanni Brea

17:30-18:00 networking / posters / drinks

Abstracts:

Guillaume Bellec

Title: "Perturbation testing and measuring gradients"

Abstract: "How can we validate a brain model from in vivo data? Quantitative prediction of neural networks can hardly reach the power of deep learning models, but the latter are not easily interpretable. Aware of the need for quantitative metrics that reflect a good mechanistic hypothesis, we suggest a method to combine deep learning on in vivo data to validate a mechanistic model hypothesis: the idea of "Perturbation testing" is to fit recurrent neural networks (RNNs) to in vivo spike recordings, and test which model can predict the effect of in vivo optogenetic perturbation on the same biological circuit. We tested multiple recurrent network models on in vivo data from the mouse sensory-motor cortices. While generic deep learning RNNs do not generalize to predict in vivo perturbations, modeling the right biophysical features improves the prediction of the perturbations substantially. Aware of this discovery, we will also discuss the implications when building models that are capable of predicting network perturbations. For instance, it enables indirect measurement of gradients in the biological circuit, which could be decisive in the search for the plasticity principles in the brain."

Raoul-Martin Memmesheimer 

Title: Weight versus Node Perturbation Learning in Temporally Extended Tasks: Weight Perturbation Often Performs Similarly or Better 

Abstract: Biological constraints often impose restrictions on plasticity rules such as locality and reward-based rather than supervised learning. Two learning rules that comply with these restrictions are weight (WP) and node (NP) perturbation. NP is often used in learning studies, in particular, as a benchmark; it is considered to be superior to WP and more likely neurobiologically realized, as the number of weights and, therefore, their perturbation dimension typically massively exceed the number of nodes. Here, we show that this conclusion no longer holds when we take two properties into account that are relevant for biological and artificial neural network learning: First, tasks extend in time and/or are trained in batches. This increases the perturbation dimension of NP but not WP. Second, tasks are (comparably) low dimensional, with many weight configurations providing solutions. We analytically delineate regimes where these properties let WP perform as well as or better than NP. Furthermore, we find that the changes in weight space directions that are irrelevant for the task differ qualitatively between WP and NP and that only in WP gathering batches of subtasks in a trial decreases the number of trials required. This may allow one to experimentally distinguish which of the two rules underlies a learning process. Our insights suggest new learning rules that combine for specific task types the advantages of WP and NP. If the inputs are similarly correlated, temporally correlated perturbations improve NP. Using numerical simulations, we generalize the results to networks with various architectures solving biologically relevant and standard network learning tasks. Our findings, together with WP’s practicability, suggest WP as a useful benchmark and plausible model for learning in the brain.

Bojian Yin

Title: Learning Deep Representations through Layer-wise Variational Alignment

Abstract: In most deep learning systems, all layers are updated together through backpropagation, a process that is powerful but biologically implausible and often difficult to interpret. In contrast, many natural systems learn locally, where each layer or region improves itself using only nearby signals. We introduce Stochastic Variational Projections (SVP), a framework that enables deep networks to learn in a fully layer-wise, local manner while maintaining strong overall performance. SVP injects controlled randomness into each layer’s activations and aligns its statistical structure with that of the previous layer, allowing each layer to refine its own representation without global coordination.  Because layers can be trained in parallel and without global coordination, SVP scales naturally to deep networks and large datasets. This approach not only offers better interpretability but also opens the door to more modular, adaptable, and efficient AI systems inspired by how biological brains learn.

Johanni Brea

Title: Two-factor synaptic consolidation reconciles robust memory with pruning and homeostatic scaling

Abstract: Memory consolidation involves a process of engram reorganization and stabilization that is thought to occur primarily during sleep through a combination of neural replay, homeostatic plasticity, synaptic maturation, and pruning. From a computational perspective, however, this process remains puzzling, as it is unclear how the underlying mechanisms can be incorporated into a common mathematical model of learning and memory. Here, we propose a solution by deriving a consolidation model that uses replay and two-factor synapses to store memories in recurrent neural networks with sparse connectivity and maximal noise robustness. The model offers a unified account of experimental observations of consolidation, such as multiplicative homeostatic scaling, task-driven synaptic pruning, increased neural stimulus selectivity, and preferential strengthening of weak memories. The model further predicts that intrinsic synaptic noise scales sublinearly with synaptic strength; this is supported by a meta-analysis of published synaptic imaging datasets.