“We only have to play three rounds of this game to understand entropy completely. [With a marble under one of three cups, shuffles the cups multiple times at high speed.] Now you probably completely lost track of where the marble is. You just went from [a probability distribution with unit mass at one of three locations]—in your head you knew exactly where the marble was—to [a probability distribution with one-third mass at each of the three locations]. That’s what a positive change in entropy looks like. Now in the second round I will go a bit slower. [Places the marble under one cup and slowly shuffles the cups a couple of times.] You probably were able to track the marble. You just went from [a probability distribution with all the mass at one location] to [a probability distribution with all the mass at a different location]. That’s what zero change in entropy looks like. The probability distribution is just as narrow as it was before. Now, in the last round, I will not show you in which cup the marble is, and now [shuffles cups slowly a couple of times], after shuffling, and after not knowing in which cup the marble was to begin with, if you now confidently knew in which cup the marble is, that would be a negative change in entropy [depicted as a distribution with three equal masses changing to a distribution with all mass at a single location], and the point where you broke the Second Law of Thermodynamics. There’s an important nuance here though that has been lingering in my thoughts for a while, and I was only recently able to fully integrate it into my worldview. Entropy is not a property of the fundamental reality, whatever that is. It is instead a measure of our lack of precision to specify reality’s actual state. If I go so fast that you can’t keep track of the marble, in reality nothing changes. The marble is still always just under one cup. It’s just you who lost track of it. The difference is in your head, in your perception of reality. Similarly, if I perceive something as going from a crystalline phase to a liquid phase, microscopically the entropy is not actually changing. Entropy is not a property of this microscopic world. Each atom always has a position and a velocity, just like the marble is always just under one cup. Nothing changes about that. The precision of my description of reality, that is what’s changing. If I describe something as crystalline, that’s a pretty narrow description. It implies very specific distances between the atoms and so on. Only very few of the possible atomic arrangements are in line with that description. Liquid on the other hand essentially translates to ‘now I’ve completely lost track of where the atoms are but they must be in that blob somewhere’. That’s way broader. Entropy measures how broad, how unspecific our descriptions are.”
−Tobias Lemke as MarbleScience, in his YouTube short video “Cracking the Mystery of Entropy with a Game”
27 January 2025
2 February 2025
I think this perfectly exemplifies the absurdity and self-torture of trying to use a single probability distribution to characterize knowledge. I assure you, MarbleScience, that changing from a crystal to a liquid is a physical change in reality, not merely a perception. If state changes of matter were not physically real, steam engines wouldn’t work unless we were paying attention to them. There is a better way to express what is known. The starting ‘distribution’ in the third round is not three equal masses of one third at each of three locations, but rather three imprecise masses at each location. Each of these is an interval between zero and one corresponding to your ignorance about whether the marble is there. Same for the ending distribution in the second round. We might such an interval the ‘dunno’ interval because we don’t know whether it’s a zero or a one. That’s what needed to represent our ignorance here. The result is called an imprecise probability distribution, which is essentially a set of many possible distributions. A single probability distribution cannot really characterize ignorance about the marble's location. And entropy is not a full description of all the uncertainty. Entropy is not a measure of how unspecific our descriptions are, but instead a characterization of physical reality. It is interpreted as the evenness of the distribution of the actual states of the system’s components over their possible states. Some analysts suggest complementing Shannon entropy with a discrete or continuous version of the Hartley measure of ignorance to create a fuller description that includes both this evenness and our ignorance.
2 February 2025
This is a great comment. It's as if the video is asserting a "Bell's Theorem" approach to reality based on ignorance rather than on the non-existence of unmeasurable properties.
Quantum mechanics is confusing enough. Let's not complicate entropy more than necessary by needlessly incorporating the mindset of the observer.
Again, great comment, and I agree completely.
2 February 2025
The reason why macroscopically processes evolve towards high entropy is just simple statistics. If for example one macrostate comprises 1000000 microstates and another just 2, it is simply very likely to end up in one of the 1000000 microstates of the first macrostate. It is the same reason why we usually don’t win in a lottery, and sure, it doesn’t matter if you are watching the process or not.
It is not exactly a steam engine, but I encourage you to watch my video about the entropy driven car: https://youtu.be/PPWEmjnqsF8
3 February 2025
@ That's a great video! Thank you for posting. You've gained a new subscriber! I hope your job search goes well.