Growth transform neural network

Growth Transform neural network is a spiking neuron and population model developed by the Adaptive Integrated Microsystems laboratory at Washington University in St. Louis. Check back for more updates as we keep adding new findings.

Background:

A single biological neuron is not optimized for energy efficiency. In fact, it consumes ~10^(-10) W for generating spikes or action potentials at an average firing rate of 10 Hz. In contrast, a highly optimized TPU/GPU consumes only ~10^(-14) W per floating-point operation per second. In spite of this, biological networks are marvels of energy-optimized systems. The human brain can rival the performance of our best deep learning hardware in cognitive tasks, all the while consuming ~20 W - a minute fraction of their power. Many neuromorphic hardware that emulate neural dynamics in biological networks have been developed in attempts to bridge this gap, but the extraordinary performance and power efficiency of our brain remains elusive.

Our approach:

In neuromorphic machine learning, neural networks are designed in a bottom-up fashion - starting with the model of a single spiking neuron (Hodgkin-Huxley, Izhikevich, or leaky Integrate-and-fire) and connecting them together to form a network. Although these spiking neural networks use biologically relevant neural dynamics and learning rules, they are not optimized w.r.t. a network function.

In contrast, traditional machine learning designs neural networks in a top-down manner, starting from a network objective (usually a loss function specified at the output layer), and reducing it to adjust parameters of a non-spiking neuron model with static non-linearities (sigmoid, tanh, etc.). Although these are highly optimized w.r.t. a network objective (usually a training error), they do not usually incorporate biologically relevant neural dynamics.

We hypothesize that a biological network can be modeled as an unified dynamical system, where the spiking activity of populations of neurons results from an emergent behavior of an underlying optimization process. Our research reconciles the top-down and bottom-up approaches to design a fully coupled neuromorphic system where the neural responses are emergent behavior of a network-level optimization, and can incorporate spike-based encoding and learning.