research concept
Motivation
Motivation
It was big news in 2017 that Alpha-Go, a computer program with deep-learning algorism running on a supercomputer with 1202 CPUs and 176 GPUs, defeated a world champion of the Go game. People said this was a dawn of the AI era, but the Alpha Go consumed 200kW just for the Go game. In contrast, the brain of the Go champion could think what the menu of today's dinner even while playing the game with only 20W power consumption. What was the real winner? It was the brain without a doubt.
It was big news in 2017 that Alpha-Go, a computer program with deep-learning algorism running on a supercomputer with 1202 CPUs and 176 GPUs, defeated a world champion of the Go game. People said this was a dawn of the AI era, but the Alpha Go consumed 200kW just for the Go game. In contrast, the brain of the Go champion could think what the menu of today's dinner even while playing the game with only 20W power consumption. What was the real winner? It was the brain without a doubt.
Why does "AI" consume immense energy?
Why does "AI" consume immense energy?
The present "AI" is based on a computer algorithm so-called "Deep Learning". It is a kind of machine learning algorithm for a particular type of the artificial neural network (a perceptron with multiple layers). We need to design the architecture to match the algorithm, and the components ("neuron" and "synapse") have to fit the architecture. The conventional digital device such as CMOS can fit the purpose very well, although the brain is never digital. Only recently, there are some challenges to adopt analogue memory devices to the system but not satisfactory.
The present "AI" is based on a computer algorithm so-called "Deep Learning". It is a kind of machine learning algorithm for a particular type of the artificial neural network (a perceptron with multiple layers). We need to design the architecture to match the algorithm, and the components ("neuron" and "synapse") have to fit the architecture. The conventional digital device such as CMOS can fit the purpose very well, although the brain is never digital. Only recently, there are some challenges to adopt analogue memory devices to the system but not satisfactory.
This top-down system is bio-inspired but quite different from a real brain. Because of the difference, it consumes significant energy during the learning process. It requires a large amount of logic calculation to search for the best (optimum) combination of the synaptic weight for the considerable amount of data set (so-called big data). However, our brain does not do such calculations thus consumes negligible energy.
This top-down system is bio-inspired but quite different from a real brain. Because of the difference, it consumes significant energy during the learning process. It requires a large amount of logic calculation to search for the best (optimum) combination of the synaptic weight for the considerable amount of data set (so-called big data). However, our brain does not do such calculations thus consumes negligible energy.
Our approach is the opposite.
Our approach is the opposite.
We considered that the alternative approach is more promising for future edge computation. It is the so-called bio-mimetic approach; i.e., a bottom-up method to construct a neuromorphic computing system. We first mimic each component of a brain by single or a set of the functional electronic device(s). Then, we design architecture suitable for the devices. The approach is indeed following how the nervous system evolved in the ancient seawater.
We considered that the alternative approach is more promising for future edge computation. It is the so-called bio-mimetic approach; i.e., a bottom-up method to construct a neuromorphic computing system. We first mimic each component of a brain by single or a set of the functional electronic device(s). Then, we design architecture suitable for the devices. The approach is indeed following how the nervous system evolved in the ancient seawater.
One of the examples done by this project is an artificial neuron. It is a field-effect transistor of a SrTiO3 channel with HfO2/Parylene double-layer gate insulator. The device shows the behaviour of "leaky-integration" pretty similar to that of the biological neuron. We are now adding another function of "firing" to the device to mimic the biological leaky-integrate and fire (LIF) neuron. We are also working on fabricating the artificial synapse, which shows the behaviour of the spike-timing-dependent plasticity (STDP).
One of the examples done by this project is an artificial neuron. It is a field-effect transistor of a SrTiO3 channel with HfO2/Parylene double-layer gate insulator. The device shows the behaviour of "leaky-integration" pretty similar to that of the biological neuron. We are now adding another function of "firing" to the device to mimic the biological leaky-integrate and fire (LIF) neuron. We are also working on fabricating the artificial synapse, which shows the behaviour of the spike-timing-dependent plasticity (STDP).
For the bio-mimetic LIF neurons and STDP synapses, a suitable neural network is, of course, a bio-mimetic one. It is the spiking neural network (SNN) with electric spikes (pulses) transmitting the signals.
For the bio-mimetic LIF neurons and STDP synapses, a suitable neural network is, of course, a bio-mimetic one. It is the spiking neural network (SNN) with electric spikes (pulses) transmitting the signals.
Our idea is that SNN shows a kind of spontaneous symmetry breaking like a phase transition, and the order is a spatiotemporal pattern called "dynamic attractor". In the quantum annealing computation method, the large-scale optimisation problem of the synaptic weight is transferred to a physical model, which shows a phase transition to give an optimum answer. Same is valid in the dynamical attracter. We think the spatiotemporal pattern formation also works for solving the optimisation problem.
Our idea is that SNN shows a kind of spontaneous symmetry breaking like a phase transition, and the order is a spatiotemporal pattern called "dynamic attractor". In the quantum annealing computation method, the large-scale optimisation problem of the synaptic weight is transferred to a physical model, which shows a phase transition to give an optimum answer. Same is valid in the dynamical attracter. We think the spatiotemporal pattern formation also works for solving the optimisation problem.
Because the phase transition is a natural providence, we do not need to do the heavy logic calculation. This is similar to what the brain computes in small power.
Because the phase transition is a natural providence, we do not need to do the heavy logic calculation. This is similar to what the brain computes in small power.
Revolution
Revolution
The future edge devices “learn” our unconscious behaviours and “infer” a bit of advice to improve the quality of our life. However, it is not a good idea to connect the edge devices to the internet, because the internet is a significant leak source of personal information.
The future edge devices “learn” our unconscious behaviours and “infer” a bit of advice to improve the quality of our life. However, it is not a good idea to connect the edge devices to the internet, because the internet is a significant leak source of personal information.
Therefore, we are awkward to use the present “AI” in cloud computers, even if the internet connection is getting faster and faster these days.
Therefore, we are awkward to use the present “AI” in cloud computers, even if the internet connection is getting faster and faster these days.
The edge device alone should do the realtime learning of the unconscious behaviours without the massive power consumption and make the proper inference. Our platform using a spatiotemporal pattern formation in SNN will become a suitable prototype of the future device. The device may behave as a genuinely reliable partner who knows our characteristics very well. The device will give us comfortable and timely support. However, the device cannot reveal what personal information is because the information is in the spatiotemporal shape, i.e., attractor.
The edge device alone should do the realtime learning of the unconscious behaviours without the massive power consumption and make the proper inference. Our platform using a spatiotemporal pattern formation in SNN will become a suitable prototype of the future device. The device may behave as a genuinely reliable partner who knows our characteristics very well. The device will give us comfortable and timely support. However, the device cannot reveal what personal information is because the information is in the spatiotemporal shape, i.e., attractor.