Embedded Files
Universality of echo-state networks
A dynamical system comprises a collection of states, and a transition law transforms states from one to another. Almost every natural dynamic phenomenon, engineering systems as well as computer algorithms can be presented as a dynamical system. When a dynamical system is measured it generates data.
data_driven_horizontal_b.pdfThe art of data-driven discovery of dynamics is to uncover the underlying dynamics from a stream of data. An important necessity for a good reconstruction is that the data reflects a complete information about the underlying system [1,2]. In other words, it represents an embedding of the dynamical system. In most practical situations one only has a partial measurement of the phase space Ω. In that case one employ several techniques to reinforce the data and make it complete.
One of the most reliable and proven techniques for embedding is delay-coordinates [5]. To establish its universality, one requires a notion of "almost every" for the infinite dimensional world of dynamical systems. This is based on the notion of prevalence [4]. Yet another technique that is reliable is the use of echo-state networks [3]. IT indirectly reinforces the data by creating an image of the dynamical system within the internal states of an artificial machine, called echo-state network (ESN) or reservoir computer. This technique has been applied succesfully to many different kinds of dynamics, but its universality is yet to be proven rigorously.
Future directions.
1) Implementing codes in Julia language.
2) Determining a notion of "almost-everywhere" and "null" for Banach manifolds, using ideas from [6].
3) Proving a universal embedding theorem for echo-state networks.
Prerequisites.
1) Foundational mathematics : familiatity with proofs and mathematical rigor; set-theory
2) Category theory : functors, categories, natural transformations
3) Analysis : continuity and differntiability, limits and convergence
4) Measure theory : probability measures, absolute continuity, Markov transitions, Borel sets, abstract Measure theory
5) Dynamical systems : orbit, invariant measure. fixed points, invariant sets
6) Differemtial calculus : derivatives, Taylor series, manifolds, tangent bundles
Sources
[1] Learning Theory for Dynamical Systems - T Berry, S Das (common generalization of ESNs and delay-coordinates)
[2] Limits of Learning Dynamical Systems - T Berry, S Das (paper presenting the need for establishing the universality of ESNs)
[3] Regularity of invariant graphs for forced systems - J Stark ("invariant graph theory" - the foundational principle for echo-state networks)
[4] Prevalence: a translation-invariant “almost every” on infinite-dimensional spaces - T Sauer, B Hunt, J Yorke (introduction to the concept of prevalence and "shy")
[5] Embedology - T Sauer, J Yorke, M Casdagli (proof of the universality of the technique of delay-cooridnates)
[6] The concept of nullity in general spaces and contexts - S Das (introduction to the most generalized notion of nullity in arbitrary spaces)
Page updated
Google Sites
Report abuse