Goals

The explosion of the success of deep learning (DL) over the past decade can be partly attributed to the open-source movement. Recent advances in open silicon and remote access to in vitro neurons can build bridges across the interdisciplinary developments within neuromorphic engineering. We aim to use these advances to:

• port open silicon (hardware) to neuromorphic engineering,

• improve the accessibility of in-vitro neural networks (wetware) to neuromorphs, and

• model and train those with spiking neural networks using neuromorphic software.

Reducing the gaps between tooling in neuromorphic hardware, software, and wetware, and learning how these three talk to each other can improve the longevity and effectiveness of future neuromorphic systems. 

Projects

We aim to use open source tools to accelerate growth in neuromorphic research. This involves building upon pre-existing tools, showing off their many capabilities, and interfacing them together.

TNT: Tiny Neuromorphic Tape-Out

Each participant can claim 100μm x 100μm of space on a chip, place their own neuromorphic design on it, and we will sponsor having it fabricated into real silicon. If you have never designed a chip before, don't sweat it! We will guide you through the full design cycle of a chip. We will end up with a design that contains a rich, heterogeneous pool of diverse neurons, learning rules, and anything else that tickles your neuromorphic creativity.

BrainDish Learning

Did you hear about BrainDish, the organoid that learnt how to play the 1970s classic arcade game 'Pong'? We will have remote access via Cortical Labs to play around with their setup. Reverse-engineer the learning rule of in-vitro brain cells in a dish, and control the stimulation protocol to form real-time, closed-loop environments.

HLS4NM: High-Level Synthesis for Neuromorphic Models

Hack together an SNN on an FPGA, and build an SNN compiler to port pre-trained neural nets for real-time processing. Bonus points if we can connect an event-camera and directly stream data in.

Embedded Real-Time Vision Processing on Speck

Take a scenic hike through the hills of Telluride in the name of science. Use Synsense's event-drive system-on-chip (SoC), "Speck", on a bodycam to build a terrain map. Or use Speck to design a motor control network. If it's real-time and in the wild, throw Speck at it.

Embedded Real-Time Audio Processing on Xylo

Synsense's "Xylo" SoC will let us throw audio processing and low-dimensional data-processing into the wild. Voice guided labelling to perform online learning, or build an Andreas Andreou voice classifier that applies feature extraction to the dulcet and calming timbre of his voice.

In-Memory Compute Compilers

In the chip world, memory circuits are the backbone of synaptic weight storage. Build the first RRAM memory compiler using OpenRAM's backend, or harness SRAM memories from OpenRAM to build in-memory compute accelerators.

Build an 8-Bit SNN Accelerator on a Breadboard

You've seen the 8-bit breadboard CPU (link). Now it's time to build an 8-bit SNN accelerator.

Materials, Equipment, and Tutorials:

Hardware

• FPGAs (Zedboard - Zynq 700; Kria KV260 Vision AI Kit)

• Speck SoC

• Xylo SoC

• Miscellanous lab / hardware equipment + Analog Discovery Kit 2

• Various event cameras (DVS128; Prophesee EKV4)

Hardware Tutorials:

• Digital Design Guide with Tiny TapeOut

• Open-Source Silicon Interactive Tutorials

• eFabless Open MPW Walkthrough Video

Software

• Python + Spiking Neural Network simulator of your taste (e.g., snnTorch, Rockpool)

• Neuromorphic DataLoading tools (e.g., Expelliarmus, Tonic)

• OpenLane Tools

• Xilinx Vivado

Software Tutorials:

• How to Train Spiking Neural Networks with snnTorch Interactive Tutorials

• Neuromorphic Data Manipulation with Tonic

Relevant Literature:

Hardware

Designing SNN accelerators using open silicon: 

F. Modaresi, M. Guthaus, J. K. Eshraghian, "OpenSpike: An OpenRAM SNN Accelerator", 2023 IEEE International Symposium on Circuits and Systems (ISCAS), May 2023. [arXiv]

Online Learning On-Chip at 28-nm:

C. Frenkel and G. Indiveri, "ReckOn: A 28nm sub-mm2 task-agnostic spiking recurrent neural network processor enabling on-chip learning over second-long timescales", 2022 IEEE International Solid-State Circuits Conference (ISSCC), Feb 2022. [IEEE]

Software

A tutorial on training SNNs using principles from deep learning: 

J. K. Eshraghian, et al. "Training spiking neural networks using lessons from deep learning", arXiv preprint arXiv:2109.12894. [arXiv]

Wetware

Neurons in a dish learn how to play Pong:

B. Kagan et al., "In vitro neurons learn and exhibit sentience when embodied in a simulated game-world", Neuron, 110(23), Dec. 2022. [SciDirect]