Speakers and abstracts

The free-energy principle and active inference have received a significant attention in the fields of neuroscience and machine learning. However, it remains to be established whether active inference is an apt explanation for any given neural network that actively exchanges with its environment. To address this issue, this work shows that a class of canonical neural networks of rate coding models implicitly performs variational Bayesian inference under a well-known form of partially observed Markov decision process models. Based on the proposed theory, we demonstrate mathematically and numerically that canonical neural networks—featuring delayed modulation of Hebbian plasticity—can perform planning and adaptive behavioural control in the Bayes optimal manner, through postdiction of their previous decisions. Moreover, this scheme enables us to estimate implicit priors under which the agent’s neural network operates and identify a specific form of the generative model, enabling us to predict subsequent learning and adaptive behaviours of the agent without observing neural activity or behaviours. The proposed equivalence is crucial for rendering brain activity explainable to better understand basic neuropsychology and psychiatric disorders. Moreover, this notion can dramatically reduce the complexity of designing self-learning neuromorphic hardware to perform various types of tasks.

Hitoshi Okamoto

Zebrafish as a model animal for studying active inference

Animals make decisions under the principle of reward value maximization and surprise minimization. It is still unclear how these principles are represented in the brain and are reflected in behavior. We addressed this question using a closed-loop virtual reality system to train adult zebrafish for active avoidance. Adult zebrafish have the ability to learn various adaptive behaviors, and their telencephalon has regions and neural circuits that are evolutionarily homologous to those of other vertebrates, including mammals. Analysis of the neural activity of the dorsal pallium during training revealed neural ensembles assigning rules to the colors of the surrounding walls. Additionally, one third of fish generated another ensemble that becomes activated only when the real perceived scenery shows discrepancy from the predicted favorable scenery. The fish with the latter ensemble escape more efficiently than the fish with the former ensembles alone, even though both fish have successfully learned to escape, consistent with the hypothesis that the latter ensemble guides zebrafish to take action to minimize this prediction error. Our results suggest that zebrafish can use both principles of goal-directed behavior, but with different behavioral consequences depending on the repertoire of the adopted principles.

Beren Millidge

The FEP, Predictive Coding, and Backpropagation in the Brain

Predictive Coding is a process theory derived from the FEP under Gaussian assumptions and has gained substantial influence within theoretical neuroscience as a potential algorithm the cortex could be implementing. Predictive Coding proposes that the cortex learns an unsupervised generative model of the world based on constantly minimizing prediction errors between predicted and observed sensory signals. In this talk, we will review recent progress in the field of predictive coding, with an especial focus towards the recent links that have been uncovered between predictive coding and the backpropagation of error algorithm. We will discuss in detail the conditions under which predictive and backprop converge, investigate intriguing recent results that indicate that predictive coding may in fact be performing a smarter and more efficient learning algorithm than backprop, and discuss whether these advances may provide a bridge between machine learning methods and computation in the brain.

Rafal Bogacz

Dopamine: precision of action selection or prediction error

The free-energy principle assumes that the brain reduces the difference between observed and expected sensory stimuli. Active inference theory proposes that this difference is minimized in two ways: through perception and learning (by changing the expectation to match the observation) and through action (by changing the world to match the expectation). Several models have been proposed to describe how this elegant idea could be implemented in brain circuits, and these models differ in the function assigned to dopamine. In the classical formulation, dopamine is proposed to encode the precision of action selection, such that the lower levels of dopamine result in more random choice. By contrast, in a recent model called DopAct, dopamine is proposed to encode the difference between observed and expected reward. So in the DopAct model, dopamine encodes reward prediction error (as reinforcement learning models), but in DopAct this prediction error is minimized not only by learning but also by action planning (as in active inference). This talk will provide overview of these models, and compare their predictions concerning dopamine with experimental data.

Naoki Honda

Decoding reward-curiosity conflict in probabilistic bandit task

Humans and animals are not optimal agents and often behave irrationally. They do not only rationally exploit rewards, but also explore the environment even without rewards so as to minimize uncertainty of the environment owing to their curiosity. However, the mechanism by which their curiosity is regulated has been largely unclear. Here, we developed a new decision-making model in the case of probabilistic bandit task by extending the free energy principle. This model successfully described conservative, rational, and explorative behaviors depending on the level of curiosity. Furthermore, we have developed a machine learning method to infer fluctuations in curiosity and confidence from behavioral data. Therefore, comparison between neural activities and curiosity estimated by our method could enable us to reveal the neural basis for controlling mental temporal dynamics such as conflicts between reward and curiosity.

Miguel Aguilera

When can we interpret organisms as performing Bayesian inference?

The free energy principle (FEP) states that any dynamical system, under specific conditions, can be interpreted as performing Bayesian inference upon its surrounding environment. In this talk we will critically review the assumptions required to derive the FEP in the simplest possible set of systems -- weakly-coupled non-equilibrium linear stochastic systems. Specifically, we explore (i) how general the requirements imposed on the statistical structure of a system are and (ii) how informative the FEP is about the behaviour of such systems. We will see that two requirements of the FEP -- the Markov blanket condition (i.e. a statistical boundary precluding direct coupling between internal and external states) and stringent restrictions on its solenoidal flows (i.e. tendencies driving a system out of equilibrium) -- are only valid for a very narrow space of parameters. Suitable systems require an absence of perception-action asymmetries that are highly unusual for living systems interacting with an environment, which are the kind of systems the FEP explicitly sets out to model. More importantly, we observe that a mathematically central step in the argument, connecting the behaviour of a system to variational inference, relies on an implicit equivalence between the dynamics of the average states of a system with the average of the dynamics of those states. This equivalence does not hold in general even for linear systems, since it requires an effective decoupling from the system's history of interactions. These observations are critical for evaluating the generality and applicability of the FEP and indicate the existence of significant problems of the theory in its current form. These issues suggest that more development is needed before the FEP, as it stands, can be applied to the kind of complex systems that describe living and cognitive processes.

Jun Tani

Cognitive Neurorobotics Study Using the Free Energy Principle

The focus of my research has been to investigate how cognitive agents can develop structural representation and functions via iterative interaction with the world, exercising agency and learning from resultant perceptual experience. For this purpose, my team has investigated various models analogous to predictive coding and active inference frameworks. For the last two decades, we have applied these frameworks to develop cognitive constructs for robots. The current talk introduces a set of emergent phenomena which we found in our recent robotics experiments. These findings could inform us of possible non-trivial cognitive mechanisms in the brains.

Ken-ichi Amemori

Neuroscientific and computational bases of the hierarchical network of anxiety in primates

Lifetime prevalence estimates of anxiety disorders and major depressive disorders (MDD) are nearly 25%. They are often accompanied by sleep disturbances, increased risk of suicide, suggesting significant social losses. The circuits related to these disorders consist of a large-scale brain network, and the disturbance of the network could induce therapeutic effects. The predictive coding framework, derived from the free-energy principle, gives us theoretical guidance on how the large-scale network produces its function. First, we introduce our recent experimental results of the anatomical and functional connectivity of the large-scale network. We performed experiments in macaques to demonstrate a causal involvement of sites in the large-scale brain network in inducing pessimistic decision-making that underlies anxiety (1, 2). We have made a series of methodologic advances to identify such sites, including the combination of causal evidence for behavioral modification of pessimistic decisions with viral tracing methods (3). Microstimulation of localized sites within the large-scale network induced pessimistic decision-making by the monkeys, supporting the idea that the focal activation of these regions could induce an anxiety-like state. Based on this experimental evidence, I will explain the potential computational framework of the large-scale network by the theoretical guidance of predictive coding. Our findings suggest that the large-scale network nodes share similar features between humans and macaques in response to conflict decisions (4). We further found that the network, including the downstream subcortical structure, consists of an actor-critics architecture, a computational reinforcement learning scheme (5). Through a series of studies, we have identified the MDD network that causes pessimistic value judgments in primates.

1. Amemori, K. & Graybiel, A. M. 2012. Nature Neuroscience 15: 776–785.

2. Amemori, K., et al. 2018. Neuron 99: 829-841.

3. Amemori, S., Amemori, K., et al. 2020. Eur J. of Neuroscience 51: 731 – 741.

4. Ironside, M., Amemori, K., et al. 2020. Biological Psychiatry 87: 399 – 408.

5. Friedman, A., Homma, D., Gibb, L., Amemori, K., et al. 2015. Cell 161: 1320-1333.

Rosalyn Moran

Active Inference from Neurobiology to Blackjack

The theory of Active Inference proposes that all biological agents retain self-ness by minimizing their long-term average surprisal. In information theoretic terms, Free Energy provides a soluble approximation to this long-term surprise ‘now’ and necessitates the development of a generative model of the environment within the agent itself. The minimization of this quantity via a gradient flow is purported to be the purpose of neuronal activity in the brain and thus provides a mapping from brain activity to their first-principle computations.

In this talk I will outline the theory of Active Inference and describe how discrete and continuous-time systems that perceive and act can be built in silico, while providing evidence for these implementations in neurobiological and behavioral recordings. Using experiments in human participants and in silico, that comprise discrete decision making over policies we will demonstrate belief based active inference and compare this to reward driven behaviors.