Generalized Operational Perceptrons (GOPs)
Generalized Operational Perceptrons (GOPs)
Embedded Files
Tutorial on Generalized Operational Perceptrons in IEEE ICIP 2021
PyGOP supports both single machine and cluster environment using CPU or GPU.
PyGOP supports both single machine and cluster environment using CPU or GPU.
What is a Generalized Operational Perceptron?
What is a Generalized Operational Perceptron?
Generalized Operational Perceptron is an artificial neuron model proposed to replace the traditional McCulloch-Pitts neuron model. While standard Perceptron model only performs a linear transformation followed by non-linear thresholding, GOP model encapsulates a diversity of both linear and non-linear operations (with traditional Perceptron as a special case). Each GOP is characterized by learnable synaptic weights and an operator set comprising of 3 types of operations: nodal operation, pooling operation and activation operation. The 3 types of operations performed by a GOP loosely resemble the neuronal activities in a biological learning system of mammals in which each neuron conducts electrical signals over three distinct operations:
Modification of input signal from the synapse connection in the Dendrites.
Pooling operation of the modified input signals in the Soma.
Sending pulses when the pooled potential exceeds a limit in the Axon hillock.
By defining a set of nodal operators, pooling operators and activation operators, each GOP can select the suitable operators based on the problem at hand. Thus learning a GOP-based network involves finding the suitable operators as well as updating the synaptic weights.
We have proposed Progressive Operational Perceptron (POP) algorithm to progressively learn GOP-based networks. We have also proposed the Heterogeneous Multilayer Generalized Operational Perceptron (HeMLGOP) algorithm and its variants (HoMLGOP, HeMLRN, HoMLRN) to learn heterogeneous architectures of GOPs with an efficient operator set search procedure. We also proposed a fast version of POP (called POPfast) together with memory extensions POPmemO, POPmemH that augment POPfast by incorporating memory path.
Subset Sampling For Progressive Neural Network Learning
Subset Sampling For Progressive Neural Network Learning
Progressive Neural Network Learning is a class of algorithms that incrementally construct the network's topology and optimize its parameters based on the training data. While this approach exempts the users from the manual task of designing and validating multiple network topologies, it often requires an enormous number of computations. We propose to speed up this process by exploiting subsets of training data at each incremental training step. Three different sampling strategies for selecting the training samples according to different criteria are proposed and evaluated. We also propose to perform online hyperparameter selection during the network progression, which further reduces the overall training time.
References
References
The list provided in the following may be incomplete. The complete list of papers related to this topic can be found in the lists of journal papers and conference papers.
D.T. Tran, M. Gabbouj and A. Iosifidis, "Subset Sampling For Progressive Neural Network Learning", IEEE International Conference on Image Processing, 2020
S. Kiranyaz, T. Ince, A. Iosifidis and M. Gabbouj, “Progressive Operational Perceptrons”, Neurocomputing, vol. 224, pp. 142-154, 2017
D.T. Tran, S. Kiarnyaz, M. Gabbouj and A. Iosifidis, "Heterogeneous Multilayer Generalized Operational Perceptron", IEEE Transactions on Neural Networks and Learning Systems, (Early Access) DOI: 10.1109/TNNLS.2019.2914082, 2019
D.T. Tran, S. Kiranyaz, M. Gabbouj and A. Iosifidis, “Progressive Operational Perceptrons with Memory”, Neurocomputing, In Press (D.O.I.: 10.1016/j.neucom.2019.10.079, 2019D.T. Tran, S. Kiranyaz, M. Gabbouj and A. Iosifidis, “PyGOP: A Python Library for Generalized Operational Perceptron”, Knowledge-Based Systems, vol. 182 (104801), 2019
D.T. Tran, S. Kiranyaz, M. Gabbouj and A. Iosifidis, "Knowledge Transfer For Face Verification Using Heterogeneous Generalized Operational Perceptrons", IEEE International Conference on Image Processing, 2019
D.T. Tran and A. Iosifidis, “Learning to Rank: A progressive neural network learning approach”, IEEE International Conference on Acoustics, Speech, and Signal Processing, 2019
D.T. Tran, J. Kanniainen, M. Gabbouj and A. Iosifidis, “Data-driven Neural Architecture Learning for Financial Time-series Forecasting”, Digital Image and Signal Processing, 2019
S. Kiranyaz, T. Ince, A. Iosifidis and M. Gabbouj, “Generalized Model of Biological Neural Networks: Progressive Operational Perceptrons”, INNS International Joint Conference on Neural Networks, Anchorage, Alaska, USA, 2017
Google Sites
Report abuse