Abstract

A recent branch of AI or Neural Networks that can handle time-varying signals often in real time has emerged as a new direction for signal analysis.  These dynamical systems are usually referred to as reservoir computers.  A central question in the operation of these systems is, why a reservoir computer (RC), driven by only one time series from a driving or source system of many time-dependent components, can be trained to recreate all dynamical time series signals from the drive  leads to the idea that the RC must be internally recreating all the drive dynamics or attractor. In addition, there have been some speculations that RCs may be a fundamental type of system that describes how neuronal networks in biology process sensory input. This has led to the possibility that the RC is creating an embedding (in the mathematical sense) of the drive attractor in the RC dynamics. There have been some mathematical advances that move that argument closer to a general theorem. However, for RCs constructed from actual physical systems like interacting lasers or analog circuits or possibly actual neuronal networks, the RC dynamics may not be known well or known at all.  And many of the existing embedding theorems have restrictive assumptions on the dynamics.  We first show that the best way to analyze RC behavior is to first treat it properly like a dynamical system, which it is.  This will lead to some conflict with existing ideas about RCs, but also a clarification of those ideas.  Secondly, in the absence of complete theories on RCs and attractor embeddings, we show several ways to analyze the RC behavior to help understand what underlying processes are in place, especially regarding if there are good embeddings of the drive system in the RC. We show that statistics we developed for other uses can help test for homeomorphisms and diffeomorphisms between a drive system and the RC by using the time series from both systems.  These statistics are called, respectively, the continuity statistic and the differentiability statistic and they are modeled on the mathematical definition of a continuous and/or differentiable function. We show the interplay of dynamical quantities (e.g. Lyapunov exponents, Kaplan-Yorke dimensions, generalized synchronization, etc.) and embeddings as exposed by the continuity statistic and other statistics based on ideas from nonlinear dynamical systems theory. These viewpoints and results lead to a clarification of various currently vague concepts about RCs, such as fading memory, stability, and types of dynamics that are useful.