Markov Chain to Bootstrap Chaotic Dynamics from Chua’s Circuit

Pikmai Hui & Hahnjoo Yoon

University of Minnesota - School of Physics and Astronomy

Minneapolis, MN 55455

Spring 2015 - MXP project

Table of Content

Introduction

Goals

• • validate that Chua's circuit can be used as a tool to study chaos

  • provide experimental evidence of effective characterization of chaotic signals by a Markov chain model

  • Prove chaos behavior in Chua's Circuit by using a bifurcation diagram and Feigenbaum constant

Background

Dynamic System

A dynamic system is a rule that describes how a point moves in a geometric space in time. This geometric space is called phase space, whose dimension can be any positive integer. The points in the phase space need not to represent real coordinates in the physical world. For example, when one pendulum is attached to another that is attached to a stable ceiling, such a double-pendulum system gives [1]

are mass of , length of string attached to, and displacement angle from vertical position of the upper pendulum, and are the respective parameters for the lower pendulum attached to the upper one.

This exemplar system discribes the displacements of angles and as time goes. These two functions draw a trajectory in a phase space. The phase space has dimensions and , hence is a two-dimensional phase space. None of the points in the phase space directly represents a geometric coordinate in the physical world.

Chaos

Chaos is a set of dynamic systems that features short-term predictability and long-term randomness in observations. These systems evolve in time according to a set of characteristic equations that does not involve random variables. In other words, as long as the initial condition (IC) remains the same, the system will always evolve in the same way, arriving at the same future everytime it is restarted at the IC. Theoretically speaking, chaos is deterministic. [2]

Recalling some well-known examples of chaos such as weather predictions [3] and stock markets [4], chaotic systems seem to following some rules at start, but quickly diverges from the perceived rules to an utter randomness, contradicting the theory. From where does the observed randomness emerge?

All physical measurements in reality involve rounding off to a predefined scale. No matter how precise the device is and how fine the scale is, there is some amout of inevitable rounding error assciated to all physical measurements. This rounding error is the source of the randomness in chaos.

It turns out that all chaotic systems share one property among more: the sensitivity to IC. [2] If the systems are simulated in time, i.e. numerically integrated, with slightly perturbed ICs, they generate complete different futures. Since in practice ICs of any prediction model are constructed by physical measurements using different kinds of sensors, an uncontrolled rounding error will always perturb the IC from its true position in phase space. The result of this perturbation is the observed randomness after a certain amount of time.

Note that chaotic systems are also abserved to be nonperiodic. [2] There are attractors in phase space that bounds the circulating trajectories in a closed region. Although each circulation of trajectory lays extremely close to each other, there is no overlapping. This is because such overlapping in phase space contradict the determinstic nature of the characteristic equations of chaotic systems. If two trajectories overlap at a point in the phase space, then at the overlapping point there are two defined Jocobian from the two trajectories. Clearly this cannot happen in a deterministic system with a uniquely defined vector field.

Markov Chain

The model used in this project is a first order Markov chain model. Markov chain models a random process in state space, in which states can be thought of as labels of integers (indices). These states has no ordinal relation. That is, state 2 is no larger than state 1. The concept of "larger" or "smaller" is not well-defined in state space.

The state space contains all possible states a system of random process can occupy. A Markov chain model considers the random process as an ordered sequence of transition of the system from one defined state to another. The sequence can be written as

And the state space S is a set of state indices denoted as

Denote the transition probability to state

given a history of old sequence as [5]

In this project, the first order simplication is made, which means the transition probability only depends on the current state, but not any previous history. That is, [5]

Markov chain also enables the definition and measurement of return time of the circulating trajectories. It is defined as the number of hopping required for the system in state space to return to its original state.

A bifurcation diagram and Feigenbaum constant

One of methods to analyze the chaotic behavior of Chua's Circuit is a bifurcation diagram and Feigenbaum constant. A bifurcation diagram is a visual summary of the succession of period-doubling produced as a parameter increases or decreases. This diagram is much more complicated as the parameter increases. Feigenbum constant is a universal constant for functions approaching chaos via period doubling.

Theory

Constructing Markov Chain from Chaos

The Markov chains in this project are constructed from both experimental and synthetic data of Chua's circuit. The state space is defined from the phase space by first partitioning the phase space of a system into

blocks, then assigning each block a state index.

The trajectories in the phase space, which consist of numerous sampled data points, are transformed into sequences of states in the constructed state space. This is done by classifying data points as occupying the state block it falls into. Hence a trajectory i.e. a stream of sampled data points can be expressed as a sequence of states in which those data points reside.

One tricky issue is how to choose a good parameter N in defining the state space. This project proposes to choose N by maximizing intrinsic entropy rate of a system. The most complex trajectories produced by experimental and synthetic data are examined in this process. It is assumed that if the chosen N can express the complexity of these trajectories in state space, it will suffice in transforming other less complex trajectories.

The definition of entropy rate is deferred until the next subsection.

Complexity and Uncertainty

A dynamic system generates different phase trajectories as parameters in the system change. There may be instances in which such change in parameters causes the topological structure of the trajectory to change. These instances of change are called bifurcations of the system.

Chaotic systems also feature bifurcations. They may generate periodic trajectories with certain set of system parameters, then bifurcate to give chaotic trajectories as these parameters change. The bifurcations that transform the system from generating periodic trajectories to generating chaotic trajectories are associated with a change of qualitative complexity of the structure of trajectories in phase space. The fact is, periodic trajectories are simple, but chaotic trajectories are not.

An assumption in this project, which is tested in the experiment, is that change in complexity of the structure of trajectories in the phase space are mapped to change in uncertainty (information) of a random process in the state space.

Therefore we can catch the changes in complexity, i.e. the transitions of the system to and from chaos, by search for changes in uncertainty of the image of the system as a random process. The tool used to produce this image in this project is a first-order Markov chain.

The uncertainty in each state %$x_i$% Markov chain is quantified by the Shannon entropy of the state, [5]

A weighted sum of the Shannon entropy is used when comparing the Markov chain of different resolutions based on one system, since the underlying system remains the same as resolution changes. The resolution of Markov chain is determined This is viewed as maximizing This sum, called entropy rate, is weighted by the static probability of occupancy for each state

,

A unweighted sum of the Shannon entropy is used when comparing systems of different system parameters, since the change in branching of state transition becomes the focus. That is, the following sum is used,

Determinsitc and First Order

Chaos is deterministic, which also implies the characteristic differential equations define a vector field in the phase space. Each point in the phase space has a unique vector indicating the direction of the trajectory motion at the next time instance. This vector is associated with the uniquely defined Jocobian of the system at every point in the phase space.

In other words, the local motion of points in the phase space is completely determined by the current point. Prediction of an immediate future requires no knowledge of the Jocobians of earlier points in the history of the trajectory.

This localized dependency enables simplification of the Markov chain into only first-order chains. As defined in earlier section, first order Markov chain only consider the label of the current state, which corresponds to the local dependency of the deterministic chaos.

A bifurcation diagram and Feigenbaum constant

To analyze the chaos behavior from Chua's circuit, there are a bifurcation diagram and Feigenbaum constant.

A bifurcation diagram is a visual summary of the succession of period-doubling produced as a parameter increases or decreases. This diagram is much more complicated as the parameter increases or decreases. As a parameter is changed, the period of the output is bifurcated. It is so called as period-doubling. [8]

This is an example of a numerically calculated bifurcation diagram. It shows period doublings. As a parameter increases or decreases (x-axis), the diagram is much more complicated and exhibits chaotic behavior.

Feigenbaum constant

Feigenbaum constant is a universal constant for functions approaching chaos via period doubling. The equation is to find the ratio which is sequential changes in the parameter and compares one to the next.

Method

Experimental Apparatus

Chua's circuit is used to generate chaotic data. The implementation is adapted from [6], generally referred to as the Kennedy op-amp implementation of Chua's circuit. Below is a circuit diagram of this implementation. Non-linearity in the circuit is grouped in the Chua's diode %$N_R$%, which makes the circuit easy to keep track of.

Voltages across and current through are recorded with different resistance R. The resistance R is changed by adjusting a trimpot.

The following table lists the component used in the specific implementation in this project

The model of op-amp and that of inductor are critical in this implementation. In particular, the internal resistance of the inductor must be less than

. An inductor of internal resistance was tried, which failed to implement the circuit.

To give readers a sense of the scale of the implementation, below is a picture of the final implementation of Chua's circuit on a breadboard

to . Below is a picture of the signature of a Chua's circuit, the double-scroll signal in space on scope

Computer Simulation

The variables in Chua's circuit are analytically described by the following characteristic equations

where is a function describing the non-linear response of Chua's diode

Below is a generic visualization of

Synthetic datasets are generated by numerically integrating the system with the IC , , and . The first two thousands data points are disregarded during analysis to wait out the transient effect of the IC. The method of integration chosen is the explicit Dormond-Prince method [7].

Datasets are collected by numerical integration with the resistane R ranging from

to with steps. Other system parameters conform to the experimental setup.

Below is a three-dimentional time-series generated by setting

Below is a casual visualization of this data in three-dimensional phase space

Results

Detailed discussions are avoided to give space for exploration for new students who are interested in this project.

To show a sense of how trajectories change as R changes, below is an animated shift of trajectories in synthetic data at different resistance R in

space.

Below are the result of plotting entropy rate of the Markov chain experimental and synthetic double-scroll data using different N.

Below are the plots of unweighted total Shannon entropy against R for experimental and synthetic Markov chains

Note that the trends not only remsemble each other, they catch two features:

• • singularities: the complexity of trajectory suddenly decreases when R crosses certain random values in both experimental and synthetic data. These values are hard to find by brute fource human inspection.

  • range of complexity: there is a range of R where the system generates significantly more complex trajectories. This range is identified automatically by Markov chain in both experimental and synthetic data.

Below are the plots of return time against R for experimental and synthetic Markov chains

Note that while the sharpness of the range of complexity is reduced in the trend, the standard deviations of the return time seem to correlate well with the complexity of trajectory. In additional, return time enables estimations of location of attractors, which are approcimated by the centers of the loops recorded defined by the calculation of return time. Below is one example of the use of this estimation on experimental data shown in

phase space.

The bifurcation diagrma of R(kΩ) vs. Amplitude of V_(C_1 ). The number of varied resistance is 36 between 1.7052kΩ and 1.5990kΩ. Each period: P_n at 1.6950kΩ (±0.001), P_(n+1)=1.6771kΩ (±0.001), and P_(n+2)=1.6730kΩ (±0.001).

Mirelle E. Broucke's bifurcation diagram with a parameter for Chua's circuit. [9]

Measured Feigenbaum constant:

Conclusion

This project has shown empirically that Markov chain can be used to estimate the complexity of trajectories produced by Chua’s circuit. Markov chain can also automatically and quickly locate the regime where trajectories are significantly more complex. A method to estimate the locations of attractors using the information of return times and loops in Markov chain is also demonstrated on a double-scroll trajectory in the experimental data.

On the other hand, the physical implementation of Chua’s circuit fails to match the output of computer simulations. The experimental data has a different complex regime. Detailed comparison between experimental and synthetic data hence becomes difficult, and the quantitative validity of the Chua’s circuit as a chaotic generator remains unchecked. However, the existence of the complex regime and that of the double scroll trajectory in experimental data strongly suggest that the physical implementation has generated chaotic signals.

Future works may include collecting more experimental data to refine the scales of plots, reducing in-ideality in the circuit by replacing inductor with a gyrator, and profiling time performance of the computational methods in a parallelized implementation.

References

• 1. Fowles, Grant R., and George L. Cassiday. Analytical mechanics. Saunders college, 1999.

  2. Boccaletti, Stefano, et al. "The control of chaos: theory and applications."Physics reports 329.3 (2000): 103-197.

  3. Matsumoto, Takashi (December 1984). "A Chaotic Attractor from Chua's Circuit". IEEE Transactions on Circuits and Systems (IEEE). CAS-31 (12): 1055–1058. Retrieved 2008-05-01.

  4. Shukla, J. "Predictability in the midst of chaos: A scientific basis for climate forecasting." science 282.5389 (1998): 728-731.

  5. Cover, Thomas M., and Joy A. Thomas. Elements of information theory. John Wiley & Sons, 2012.

  6. Michael Peter Kennedy, “Robust OP AMP Realization of Chua’s Circuit, 1992”, In Frequenz,Vol 46, pages 68-80 , 1992.

  7. Bogacki, P., and Lawrence F. Shampine. "An efficient runge-kutta (4, 5) pair."Computers & Mathematics with Applications 32.6 (1996): 15-28.

  8. Bryan Prusha, "Measuring Feigenbaum's δ in a Bifurcating Electric Circuit", 1997.

  9. Michael Peter Kennedy, "Robust OP Amp Realization of Chua's Circuit, 1992", In Frequenz, Vol 46 pages 68-80, 1992.