Embedded Files
Technische Universität Graz, Austria
"Bringing reservoirs up to speed for deep learning"
"Bringing reservoirs up to speed for deep learning"
Reservoirs are good at temporal processing and easy to train. They are suitable for temporal processing because they employ a recurrent neural network or some other type of dynamical system. They are easy to train, since they only adapt linear readouts from this dynamical system for a specific task, rather than tackling the more difficult problem of training the recurrent network itself. They also appear to be more plausible as models for brain processing than many models that arise in deep learning. In deep learning and modern learning-driven AI one rarely uses reservoirs for tasks that have a temporal dimension because their performance is sub-optimal. Instead, one relies on recurrent network of LSTM (Long-Short Term Memory) units that are trained by backpropagation through time (BPTT). In addition, one applies Learning-to-Learn (L2L) methods in order to enable networks to learn from few examples. I will discuss recent progress in narrowing the performance gap between reservoir computing and deep learning for the case of spike-based reservoirs.
Wolfgang Maass is Professor of Computer Science at the Graz University of Technology in Austria since 1991. From 1992 to 2017 he has been head of the Institute of Theoretical Computer Science at the Graz University of Technology. He conducted his activity in several research institutes, including MIT, the University of Chicago, the University of California at Berkeley, the EPFL Brain-Mind Institute and others. Since 2013 he is Member of the Academia Europaea. He serves as editorial board member in major journals in the field of Machine Learning.
The University of Birmingham, UK
"Reservoirs as temporal filters and feature mappings"
"Reservoirs as temporal filters and feature mappings"
Parametrized state space models in the form of recurrent networks are often used in machine learning to learn from data streams exhibiting temporal dependencies. To break the black box nature of such models it is important to understand the dynamical features of the input driving time series that are formed in the state space. I will talk about a framework for rigorous analysis of such state representations in vanishing memory state space models, such as echo state networks (ESN). In particular, we will view the state space as a temporal feature space and the readout mapping from the state space as a kernel machine operating in that feature space. This viewpoint leads several rather surprising results linking the structure of the reservoir coupling matrix with properties of the dynamic feature space.
Peter Tiňo was a Fulbright Fellow with the NEC Research Institute, Princeton, NJ, USA, and a Post-Doctoral Fellow with the Austrian Research Institute for AI, Vienna, Austria, and with Aston University, Birmingham, U.K. Since 2003, he has been with the School of Computer Science, University of Birmingham, Edgbaston, Birmingham, U.K., where he is currently a full Professor - Chair in Complex and Adaptive Systems. His current research interests include dynamical systems, machine learning, probabilistic modelling of structured data, evolutionary computation, and fractal analysis. Peter was a recipient of the Fulbright Fellowship in 1994, the U.K.–Hong-Kong Fellowship for Excellence in 2008, three Outstanding Paper of the Year Awards from the IEEE Transactions on Neural Networks in 1998 and 2011 and the IEEE Transactions on Evolutionary Computation in 2010, and the Best Paper Award at ICANN 2002. He serves on the editorial boards of several journals.
Google Sites
Report abuse