21. Information Entropy

The entropy of thermodynamics (Clausius) is a state of the system but the entropy of information (Shannon) is a measure of the system but not a state of the system. The entropy of thermodynamics is actually the average of possible states of all the elements of a unit and then ascribing that average to each element. Thus the unit could be made up of ether and not molecules. This number works for understanding thermodynamic processes and calculating efficiencies. The entropy of information is based on the actual states the elements take and summing that over the unit. These two “entropies” are related since they both have to do with states of the system. The actual states of individual elements (the information entropy) is related the how we got here, while the thermodynamic entropy is a state of the system, irrespective of how we got here.

For such a system the actual states of individual elements is similar to a series of symbols lined up as though expressing something. This is a symbolic representation of the information of information entropy. The average of all these states is interpretable by science as the thermodynamic entropy but information in the states is not interpretable by science but is in the bailiwick of hermeneutics. Interpreting information entropy is a hermeneutic process not a scientific process.

In a system in equilibrium the information entropy is not important. In out of equilibrium situations the information entropy is more important, particularly in emergent situations where the limitation on possible energy states is very small. It is noteworthy that such situations have a very limited amount of randomness. Such cases can be viewed as very close too determinism on a scale from totally random to deterministic. Information entropy plays a part in the appearance of apparent patterns in chaos- complexity systems and for the same reason it has a more significant role in emergent patterns, which are more than apparent, in emergent situations.

In an emergent process the information entropy becomes very important because of the limitation on future states. The present states which can move forward into the possible states are part of the puzzle of emergence. It is not just energy levels but also state configuration that limits the transition to the limited future states.

Jerome Heath