Binarized Neural Networks

The brain achieves outstanding energy efficiency by integrating "logic" and "memory" as tightly as possible, an idea known as in-memory computing, and by relying on low-energy low-precision arithmetic. Currently, state-of-the-art Artificial Intelligence is obtained with artificial neural networks, the topology of which shares properties with the brain. However, their energy consumption is considerable is abysmal, one of the largest challenge of current microelectronics.

Reproducing the strategy of the brain is challenging, because artificial neural networks require considerable memory, and rely on extensive floating point arithmetic. Realizing in-memory computing in these conditions has unbearable cost. Currently, many research groups are investigating the in-memory implementation of neural networks using novel memory technologies, and lower precision than CPUs and GPU. In our group, we are pushing this strategy to the extreme.

Binarized Neural Networks are neural networks where synapses neurons and synapses assume only binary values, meaning +1 and -1. These networks have reduced memory requirements, and their arithmetic is ridiculously simple: in particular, floating point multiplications are by logic exclusive NOR gates! Nevertheless, these networks can achieve excellent accuracy on vision and signal processing tasks. They are, therefore, outstanding candidates for in-memory computing implementation with novel memories.

With the groups of Marc Bocquet and Jean-Michel Portal at IM2NP, Aix-Marseille Université, and Elisa Vianello and Etienne Nowak at CEA, LETI, Université Grenoble-Alpes, we are developing hardware binarized neural networks with hafnium oxide resistive memory (memristor) integrated in the back-end-of-line of a CMOS process. In particular, we are working hard on the questions of errors: emerging memory are normally used with strong Error Correcting Codes. But using them is not reasonable in an in-memory computing context, as error decoding would consume most of the energy of the systems! Therefore, we are proposing ECC-less solutions.