Research / 研究

Neural network structure reconstruction for energy efficient computation

(Representative papers: T. Leleu et al., PRE, 91(2):022804, 2015; T. Leleu et al., NOLTA, IEICE, 9(2):281–294, 2018. Other)

Reconstructing accurately the structure of neural networks from biological data is essential for the analysis of simultaneous recordings from many neurons, and, in turn, for the understanding of neural codes. Classical techniques that are based on pairwise correlations can be used for extracting “functional connections” but does not map unambiguously to a given set of physical (synaptic) connections. In order to resolve this ambiguity, we have proposed recently a method for which there is one-to-one correspondence between statistical properties of packets of spikes (or bursts) and the structure of synaptic connections. This technique can be applied to the reconstruction of synaptic connectivity with fine precision during the development of in-vitro cultures or during learning using in vivo data. The method relies on mathematical combinatorics and a similar method can be generalized to various neuronal models. We are collaborating with experimentalists that develop novel bio-probes and are analyzing recordings from cultured neuronal networks coupled to microelectrode arrays. The goal of this research is the identification of neural patterns that emerge in vitro and in vivo data, and the analysis of their advantage in terms of energy efficient computation. Longer term goals are the design of neural prostheses that reproduce as precisely as possible the firing statistics of healthy neural substrate.

Neuro-inspired algorithms for solving combinatorial optimization problems

(Representative papers: T. Leleu et al. Physical Review Letters, 122:040607, 2019; Timothee Leleu et al., Physical Review E, 95(2):022118, 2017)

We have shown recently that, contrarily to general belief, neuro-inspired systems can solve some combinatorial optimization problems (in particular Ising problems) more efficiently than state-of-the-art heuristics running on classical computers (such as the BLS algorithm). Our method is based on relaxation of binary spins to analog values and the addition of “error-encoding” neurons that allow improving the performance compared to previously proposed neural network approaches, notably by control of amplitude heterogeneity. We are extending the concepts of chaotic neural networks, that were proposed for solving the minima problem of Hopfield neural networks, and propose a robust scheme that do not get stuck in a subspace of non-optimal solution due to chaotic traps, by imposing that the entropy production rate of the system remains always positive. We are extending this method other types of problems such as satisfiability (SAT) and quadratic assignment problems (QAP), by notably utilizing technique called the valid-subspace approach. In order to explain the good performance of the proposed scheme, we are applying complexity analysis that come from the theory of spin glasses, notably the dynamical TAP approach. The neural network approach that we propose include the systematic mapping of combinatorial optimization problems, reduction of problem sizes, automated tuning of parameters of the algorithm, and automated configuration of the artificially neural network structure. This work shows that neural network approaches to combinatorial optimization are close to a paradigm shift tipping point that is similar to the one that have been taking place recently in the field of machine learning with the development of deep learning.

Engineering of neuro-inspired unconventional hardware for energy efficient computation

(Representative papers: Yoshihisa Yamamoto, Kazuyuki Aihara, Timothee Leleu, et al., Nature Partner Journal: Quantum Information, 3(1):49, 2017)

Neuro-inspired approaches to solving combinatorial optimization problems can be implemented directly on unconventional hardware that, by construction, allow faster and more efficient computation because of their lower-level organization resembles neural systems. In particular, we have implemented using Field-Programmable Gate Arrays, which is a readily commercially available hardware, the neuro-inspired method we have proposed for solving combinatorial optimization and shown that 100 to 1000 reduction in the time in solving hard problems can be achieved using this hardware. In our research we are engineering similar algorithms on emerging hardware that will very likely, in the long run, replace digital electronics for certain special-purpose computing. In particular, we have proposed an opto-electronic implementation of type I and type II spiking neural networks, and extending our models to the implementation of our schemes on integrated photonics. The development of computational schemes that solve combinatorial optimization problems using time-coding in spiking neural networks and their implementation on various neuromorphic hardware is an important aspect of our research.