Embedded Files
Following Schneidman et al.\cite{Schneidman2003}, for example, we might say that the uncertainty associated with mth-order marginals for a system of L variables is something like:
In order to consider correlations on more than one level in hierarchical complex systems, we begin with ways to quantify pair and higher-order components of total-correlation (the generalization of mutual-information to more than two subsystems as a special-case of always-positive KL-divergence\cite{Kullback51}) on a single level.
This may facilitate use of subsystem correlation-information as the natural thermodynamic limit on evolving complexity. As shown in Fig. \ref{Fig1}, for example, a limit on subsystem correlation-information in the John Conway's life adds realism to the process by preventing the steady-state endings to the evolution process.
,
from which the "connected information" of order m for a system with L variables becomes, in terms of both our equations and their equation (6):
Total correlation $I_c$ is simply the sum of these positive terms for $m$ running from 2 (pair correlations) up to L.
This unpacking of always-positive total-correlation measures into pair and post-pair components is of special interest to physicists because of the total-correlations connection, as a special case of KL-divergence, to applications for the second-law of thermodynamics. In fact the move to always-positive information-measures, like KL-divergence as the negative of Shannon-Jaynes entropy\cite{Gregory2005}, may signal a pedagogical move from entropy-1st thermodynamics to correlation-1st thermodynamics\cite{Fraundorf2011a} in the decades ahead.
Related references:
S. Kullback and R. A. Liebler (1951) ``On information and sufficiency" Annals of Mathematical Statistics 22, 79-86.
Elad Schneidman, Susanne Still, Michael J. Berry II, and William Bialek (2003) "Network information and connected correlations", Phys. Rev. Lett. 91:23, 238701 (link).
Phil C. Gregory (2005) Bayesian logical data analysis for the physical sciences: A comparative approach with Mathematica support (Cambridge U. Press, Cambridge UK) preview.
P. Fraundorf (2008) "The thermal roots of correlation-based complexity", Complexity 13:3, 16-26 abstract eprint.
Page updated
Google Sites
Report abuse