Literature
Entropy, energy and information
These notes, and the related freshman-level course, are about information. Although you may have
a general idea of what information is, you may not realize that the information you deal with can be
quantified. That’s right, you can often measure the amount of information and use general principles about
how information behaves. We will consider applications to computation and to communications, and we will
also look at general laws in other fields of science and engineering.
One of these general laws is the Second Law of Thermodynamics. Although thermodynamics, a branch of
physics, deals with physical systems, the Second Law is approached here as an example of information processing
in natural and engineered systems. The Second Law as traditionally stated in thermodynamics deals
with a physical quantity known as “entropy.” Everybody has heard of entropy, but few really understand
it. Only recently has entropy been widely accepted as a form of information.