March 27th

Prof. Dr. Wolfgang Maass

Institute of Machine Learning and Neural Computation, TU Graz

A promising research direction for SNNs

I will sketch new methods for porting higher cognitive functions, such as planning, problem solving, compositional computing, into sparsely active networks, using just local synaptic plasticity. These methods are based on new insight from neuroscience and cognitive science.

Prof. Dr. Hussam Amrouch

Chair of AI Processor Design, TUM

Algorithm/Technology Co-Optimization Unleashed: The Art of Designing AI Chips for Brain-Inspired Edge Computing

Edge computing for AI has emerged as a pivotal strategy for ensuring security and privacy, particularly in applications where personal data and sensitive biomarkers demand rigorous protection. Moreover, it profoundly helps reduce the carbon footprint associated with running AI algorithms on power-hungry GPUs. This talk explores a transformative approach that transcends traditional cloud infrastructures by executing AI algorithms directly at the end-user. We address the fundamental challenge of limited computing resources at the edge by introducing brain-inspired computational algorithms—specifically, hyperdimensional computing and spiking neural networks—that are inherently more energy efficient than classical deep learning methods. By leveraging innovative in-memory computing AI accelerators and custom RISC-V processor architectures augmented with specialized AI instructions, our work embodies a true algorithm-technology co-optimization. Experimental silicon measurements and results from our AI processor chip, fabricated in a 22 nm technology node, demonstrate both inference and training capabilities operating within a mW power envelope, thereby opening new doors for secure, efficient, and sustainable edge AI.

Prof. Dr. Friedemann Zenke

Friedrich Miescher Institute for Biomedical Research, University of Basel

Training spiking neural networks - From theory to applications

Prof. Dr. Sander Bohte

CWI ML Group, University of Amsterdam

Scaling Spiking Neural Networks

While artificial neural networks represent a highly successful mapping from neuroscience AI, and clearly capture important aspects neuronal information processing, data from experimental neuroscience strongly suggests that the ANN abstraction omits important biological computational principles. This includes such features as the pulsed nature of neural communication, the diversity and function of neuronal morphology, and the inherently time-continuous mode of operation. To investigate these computational principles, we need to be able to train large and complex networks of spiking neurons for specific tasks. I will show how effective online and approximate learning rules enable the supervised training of large-scale networks of detailed spiking neuronal models, how we can integrate extended temporal delays in a principled and efficient manner, and how these spiking neural network models can be integrated with brain-derived decision-making circuits to operate continuously. As I will argue, this approach opens up the investigation of both network and neuronal architectures based on functional principles, while at the same time demonstrating the potential power and energy efficiency of AI-solutions based on neuromorphic computing as embodied by spiking neural networks.

Prof. Dr. Gitta Kutyniok

Bavarian AI Chair for Mathematical Foundations of Artificial Intelligence, LMU

Reliable and Sustainable AI: From Mathematical Foundations to Next Generation AI Computing

The new wave of artificial intelligence is impacting industry, public life, and the sciences in an unprecedented manner. However, one current major drawback is the lack of reliability as well as the enormous energy problem of AI. In this talk, we will first discuss our theoretical results pointing towards the advantage of analog AI systems from a computability viewpoint. Focussing on spiking neural networks as the current key analog model, we will then present our analysis of their expressive power as compared to classical neural networks. 

Dr. Dominik Dold

Chair of Mathematics of Machine Learning, University of Vienna

Causal pieces: analysing and improving spiking neural networks piece by piece

Spiking neural networks (SNNs) have lately garnered attention due to their prospect of enabling low-power hardware solutions for deep learning methods, particularly for edge applications, e.g., in factories and in outer space onboard spacecraft. However, formal frameworks that allow a comparison of their capabilities in terms of performance, stability, and energy efficiency with traditional artificial neural networks (ANNs) are still lacking. In this talk, I present recent work where we introduce a novel concept for SNNs derived from the idea of “linear pieces” used to analyse the expressiveness of ANNs  to approach this challenge. We prove that the input domain of SNNs decomposes into distinct causal regions where its output spike times are locally Lipschitz continuous with respect to the input spike times and network parameters, with each region characterized by a different Lipschitz constant. The number of such regions – which we call “causal pieces” – is a measure of the approximation capabilities of  SNNs, and even single neurons can feature a high number of causal pieces that depends exponentially on the number of inputs. In particular, we demonstrate in simulation that parameter initialisations  which yield a high number of causal pieces strongly correlate with SNN training success. Moreover, we find that feedforward SNNs with purely positive weights exhibit a surprisingly high number of causal pieces, allowing them to achieve competitive performance levels on machine learning benchmarks. We believe that causal pieces are not only a powerful and principled tool for analysing and improving SNNs, but also provide a framework facilitating a direct comparison of SNNs and ANNs.

Dr. Sebastian Billaudelle 

Institute of Neuroinformatics, University of Zurich and ETH Zurich

Neuromorphic computing – between bio-emulation and machine intelligence

Neuromorphic computation is situated at the intersection of neuroscience, electronics engineering, information theory, and artificial intelligence, and spans applications from research-focused brain emulation to signal processing. This talk explores some of the underlying computational paradigms, especially from an engineering perspective, and attempts an unbiased look at the neuromorphic design space. It introduces the BrainScaleS-2 platform and highlights two distinct applications: 

The first example showcases the implementation of a complex synaptic plasticity paradigm to illustrate the use of neuromorphic computation for multi-timescale modeling. 

The second focuses on spike-based signal processing for efficient low-latency computation, employing end-to-end optimization strategies adopted from machine learning.

March 28th

Prof. Dr. Wulfram Gerstner

Laboratory of Computational Neuroscience, EPFL

SNN versus ANN: there is no performance gap!

Prof. Dr. Timothée Masquelier

Centre de Recherche Cerveau et Cognition, CNRS

Learning delays with backprop in SNNs – a new method based on Dilated Convolutions with Learnable Spacings.

Spiking neural networks (SNN) have been studied for several decades, yet interest in them has surged recently due to a major breakthrough: surrogate gradient learning (SGL). SGL allows the training of SNNs with backpropagation, just like real-valued networks, outperforming other training methods by a large margin. In my group, we have demonstrated that SGL enables the learning of not only connection weights but also connection delays. These delays represent the time needed for one spike to travel from the emitting to the receiving neurons. Delays matter because they shift the spike arrival times, which should be synchronous to trigger an output spike. Thus plastic delays significantly enhance the expressivity of SNNs. If this fact is well established theoretically, efficient algorithms to learn these delays have been lacking. We proposed one such algorithm, based on a new convolution method known as dilated convolution with learnable spacings (DCLS), which outperforms previous proposals. Our results show that learning delays with DCLS in fully connected and convolutional SNNs strongly increases the accuracy on several temporal vision and audition tasks, leading to new state-of-the-arts. Our approach is relevant for the neuromorphic engineering community, as most existing neuromorphic chips support programmable delays. In the long term, our research could also provide insights into the role of myelin plasticity in the brain.

Prof. Dr. Holger Boche

Chair of Theoretical Information Technology, TUM

TBA

Prof. Dr. Emre Neftci

RWTH Aachen

Neuromorphic Principles for Self-Attention

The causal decoder transformer is the workhorse of state-of-the-art large language models and sequence modeling. Its key enabling building block is self-attention, which acts as a history-dependent weighting of sequence elements. Self-attention can take a form strikingly similar to synaptic plasticity, which can be efficiently implemented in neuromorphic hardware.

So far, challenges in deep credit assignment have limited the use of synaptic plasticity to relatively shallow networks and simple tasks. By leveraging the equivalence between self-attention and plasticity, we explain how transformer inference is essentially a learning problem that can be addressed with local synaptic plasticity, thereby circumventing the online credit assignment problem. With this understanding, self-attention can be further improved using concepts inspired by computational neuroscience, such as continual learning and metaplasticity. Since causal transformers are notoriously inefficient on conventional hardware, neuromorphic principles for self-attention could hold the key to more efficient inference with transformer-like models.

Prof. Dr. Daniel Goodman

Neural Reckoning Group, Imperial College London

Spikes are cool again - what's next?

Recent years have seen an explosion of interest in the spiking and neuromorphic fields, after a long period where the eyes of many had begun to drift towards the promise of machine learning and artificial neural networks. Spikes are cool again! So what's next for our field? I want to zoom out and ask why spikes might be important from several different points of view: neuroscience, neuromorphic computing, machine learning, and from the point of view of the fundamental principles of intelligent systems. Of course energy efficiency looms large, but there are other possibilities that have been less well explored, or where the results are less conclusive. I'll finish by discussing some of my guesses about the key reasons spikes (or something like spikes) could be a key part of a general approach to the design of intelligent systems. I will illustrate this with some examples from my research and from other groups: on sound localisation, multimodal sensing, delay learning, sparsity and modularity.

Dr. Laura Kriener

Institute of Neuroinformatics, University of Zurich and ETH Zurich

DelGrad: Event-based gradients in SNNs for training delays and weights on neuromorphic hardware

Spiking neural networks (SNNs) inherently rely on the timing of signals for representing and processing information. Incorporating trainable transmission delays, alongside synaptic weights, is crucial for shaping these temporal dynamics. While recent methods have shown the benefits of training delays and weights in terms of accuracy and memory efficiency, they rely on discrete time, approximate gradients, and full access to internal variables like membrane potentials. This limits their precision, efficiency, and suitability for neuromorphic hardware due to increased memory requirements and I/O bandwidth demands.

To address these challenges, we propose DelGrad, an analytical, event-based method to compute exact loss gradients for both synaptic weights and delays. The inclusion of delays in the training process emerges naturally within our proposed formalism, enriching the model’s search space with a temporal dimension. Moreover, DelGrad, grounded purely in spike timing, eliminates the need to track additional variables such as membrane potentials. To showcase this key advantage, we demonstrate the functionality and benefits of DelGrad on the BrainScaleS-2 neuromorphic platform, by training SNNs in a chip-in-the-loop fashion. For the first time, we experimentally demonstrate the memory efficiency and accuracy benefits of adding delays to SNNs on noisy mixed-signal hardware. Additionally, these experiments also reveal the potential of delays for stabilizing networks against noise. DelGrad opens a new way for training SNNs with delays on neuromorphic hardware, which results in fewer required parameters, higher accuracy and ease of hardware training.

Matthias Ochs

Chair of Circuit Design, TUM

Efficient Hardware for Spiking Neural Networks Using Analog Time-Discrete Computation

With the increasing adoption of IoT devices, there is a growing demand for power- and energy-efficient hardware for artificial intelligence (AI). These devices are often battery-powered and consist of only a few sensors. Since transmitting raw data to higher computing nodes is expensive in terms of both energy and bandwidth, on-device processing is essential to extend their operational lifetime.

Spiking Neural Networks (SNNs) have emerged as a promising alternative to traditional Artificial Neural Networks (ANNs) due to their event-driven nature and energy efficiency. However, SNNs require dedicated hardware for efficient computation, as their temporal processing differs from conventional ANNs. To address this, we propose a novel analog time-discrete computation scheme that combines the benefits of analog efficiency with digital design principles.

Advantages of the Proposed Approach

1.) Zero Static Power Consumption – Unlike fully analog hardware, our approach eliminates static power dissipation while retaining the efficiency of parallel computation.

2.) Scalability with Digital Design Principles – The circuit design follows standard digital methodologies (timing, delay, fan-out, etc.), allowing for integration into automated design flows for scalability.

3.) Power-Adjustable Global Clock – A single global clock governs the system’s operation, making it possible to dynamically adjust frequency and power consumption based on application requirements.

4.) Seamless Integration with Analog Sensors – The system’s analog nature enables direct interfacing with sensors, eliminating the need for power-hungry analog-to-digital conversions and further reducing overall energy consumption.

By leveraging analog and time-discrete computing, our approach significantly reduces power consumption while preserving essential spike-based information, including spike timing and rate. This makes it well-suited for energy-efficient inference and learning in SNN-based AI applications.