Complex Systems & Method Development

What quantifiers can we use to characterise predictability, optimality, complexity and efficiency in complex systems? I am interested in investigating if the usefulness of these quantifiers is system specific, and when analytic progress can be made.

In order to evaluate models or signal architectures, it can be useful to identify an "optimal" model or signal architecture to compare with. To do so, we have to typically set an optimization goal. Model systems for these type of optimizations include scenarios where the goal is clear - such as in the bandit problem, where players want to optimize their total reward after playing the game for time T. In these cases, we can compare the player's strategy to the optimal strategy that gives the best reward. Sometimes, one can perform similar optimization with a clear optimization or loss function also in biology; then one can either derive or infer an optimal bound (or hope this has been done already).

At other times, what variable (if a single one) an organism may want to have optimized is unknown. When one has a biological intuition for what a particular signal S is for (say X), one can optimize the mutual information between this variable X and the organism's interpretation of the signal, C. One optimization scheme with performs this optimization is the information bottleneck scheme (see Tishby, Pereira & Bialek), which optimizes I(C,X) - λ I(C;S). This scheme essentially performs a compression on the data, and one can then compare models to the abstract "best" compression. The advantage of information-theoretic optimization schemes is that sometimes, optimal bounds can be derived analytically.