UMBC - Methodology

Hebbian Learning

Hebbian learning is a type of activity-dependent synaptic plasticity in which the connection between presynaptic and postsynaptic neurons is strengthened by coordinated activation of the two neurons [1]. Hebb proposed the learning principle in 1949, claiming that if a presynaptic neuron A is successful in triggering a postsynaptic neuron B repeatedly while itself (neuron A) is functioning, it would gradually grow more effective in activating neuron B. In both experimental and computational neuroscience, the notion is widely used and researched. In our project, we resort to the hebbian learning rule for updating RNN-G connectivity.

Delta Learning

In machine learning and neural network, the delta learning rule is a type of back-propagation techniques that helps to refine connections in AI networks by using layers of artificial neurons to connect inputs and outputs [2]. Back-propagation is the process of recalculating artificial neuron input weights using a gradient approach. The difference between a target activation and an actual acquired activation is used in delta learning. Network connections are modified using a linear/non-linear activation function. Gradient descent learning is accomplished using an error function. The Delta rule contrasts an output to a target. In our project, we employ delta learning for updating G-A connectivity.

Hebbian learning for spatial sequence and delta update for temporal activity

As mentioned before, we employ Hebbian learning in order to capture the input motor sequence within different excitatory clusters of RNN module. Here, in ACDC architecture, all the feed-forward motifs within the RNN module are learned through training rather than hardwired [3-5]. The Hebbian updating rule adopt for this project is as follows.

Where ~x_jis presynaptic activity low-pass filtered over a time scale 𝛕_w. x_i is postsynaptic activity; α1 and α2 are learning rate parameters. When ~x_jand xi are both simultaneously > 0, Wij goes to Wmax; otherwise Wij decreases.

Unlike the Hebbian learning used in biological plausible excitatory neuron cluster, as mentioned before, delta learning is used for the action-specific execution time, which is encoded by the weights of G-A connectivity for our ACDC network. As, we adopt the delta rule as time learning, an agent receives a supervisory signal explicitly indicating whether a specific motor sequence is produced before or after the appropriate time [4-5], and is described by following equations.

Here, the change in weight (ΔW) between the j^th G and A neurons is driven by the learning rate η, and the error computed as the difference between the observed and desired response time (t) for each action.

Neurons dynamics

Excitatory Neurons

The dynamics in the ACDC model represent the sequential expansion of the RNN-BG-thalamus-RNN loop. The loop activates a cluster of excitatory RNN neurons and Excitatory neurons in RNNs obey the following equation:

where x_i and x_j represent post- and pre-synaptic RNN unit activity and Wij is the recurrent weight matrix. J^EI and J^EA represent respectively the weights from the shared inhibitory neuron and from the motor thalamus neurons to the excitatory RNN units. x_I x_A and x_inrepresent respectively the activity of the shared inhibitory neuron, A neurons, and the input to the excitatory RNN units. γ_E is the gain on A neurons activation projected to the excitatory RNN neurons. 𝝉rnn is the encoding constant. Here, ϴ(x) is a nonlinear function which maintains the non linearity. We assume if ϴ(x) < 0, then ϴ(x) = 0.

Inhibitory Neurons

The shared inhibitory neuron of RNN cluster x_I activation is described by

where J^IE, J^IA and γ_I respectively represent weights from RNN excitatory neurons to their common inhibitory neurons, weights from a neurons to common inhibitory neurons, and increased activation of a neurons for projection of RNNs to inhibitory neurons.

G Neurons Activation

Each G cell accumulates evidence for the j^th A neurons associated to the i^thorder, following below equation:

Here, g_j is the activation of the j^th G units. Wij is the weight matrix representing connectivity between RNN and G units. x_i is the activity of the RNN excitatory units. J^GN is the inhibitory weight between the j^th N and G neurons. n_j is the activation of the j^th N G neurons, and τg is the encoding constant.

A Neurons Activation

G cells facilitate response execution by projecting towards the corresponding A neurons whose dynamics are governed by :

Here, a_j is the activation of the j^th action. g_j is the activation of the j^th G unit. b is the negative bias. Ө is a nonlinear function, and τa is the encoding constant. J^AG is the weight from the j^th G unit to the j^th A unit and it is initially randomly drawn from a gaussian distribution with mean = 2 and standard deviation. = 0.2.

N Neurons Activation

The dynamics of N G cells are in turn dictated by following equation:

Here, n_j is the activation of the j^th N G neurons. J^NA is the weight from the j^th A unit to the j^th N G unit. a_j is the activation of the j^th A neurons, and τn is the encoding constant.

References:

Munakata, Y. and Pfaffly, J., 2004. Hebbian learning and development. Developmental science, 7(2), pp.141-148.
Li, X., Xiong, H., Wang, H., Rao, Y., Liu, L., Chen, Z. and Huan, J., 2019. Delta: Deep learning transfer using feature map with attention for convolutional networks. arXiv preprint arXiv:1901.09229.
Maes, A., Barahona, M. and Clopath, C., 2020. Learning spatiotemporal signals using a recurrent spiking network that discretizes time. PLoS computational biology, 16(1), p.e1007606.
Oh, S.W., Harris, J.A., Ng, L., Winslow, B., Cain, N., Mihalas, S., Wang, Q., Lau, C., Kuan, L., Henry, A.M. and Mortrud, M.T., 2014. A mesoscale connectome of the mouse brain. Nature, 508(7495), pp.207-214.
Hunnicutt, B.J., Jongbloets, B.C., Birdsong, W.T., Gertz, K.J., Zhong, H. and Mao, T., 2016. A comprehensive excitatory input map of the striatum reveals novel functional organization. elife, 5, p.e19103.