What is the philosopher's take on information and thermodynamic entropy?

Question

So there are various interpretations of probability.

Frequentism is the likelihood of events of say for example if I roll a dice the likelihood of of getting a 5 is 1/6 if repeated over and over.

Propensity is given by how much you know about the system for example if I know the initial conditions of the dice roll well enough then I can predict the odds of a 5 is 100% (for example).

In statistical mechanics the thermodynamic entropy of a system seems to be describe the likelihood of an event but then we also have the notion of information entropy which is related to the second type of probability. Many physicists are comfortable swapping and conflating the two. Going so far that they even equate the two. (See Is information entropy the same as thermodynamic entropy? (PhysicsSE).)

What is the relationship if any according to philosophers between information and thermodynamic entropy?

Answer 1

You never have 100% predictive power. Quantum events involve fundamental uncertainties, and can in principle effect macroscopic events (eg a powerful cosmic ray changing the momentum of the dice at a critical moment, or even more uncertainly a beta decay of a carbon 14 nucleus). There are finite probabilities of landing on an edge or point also. It's good to think about probability as a way to deal with limited information, but the uncertainty principle and the observer effect (the impact of taking measurements on increasing uncertainty) mean information is always limited, in a way that I would describe as there not being such complete information meaningfully in our universe. Three-body dynamics of large interacting blackholes mean uncertainties below the Planck scale can have have macroscopic effects.

Entropy causes a number of confusions. It is relative not absolute, strictly speaking we can only measure change in it, and if there are hidden microstates or degrees of freedom calculations will be incomplete. We typically idealise systems as closed and close to equilibrium, far-from equilibrium thermodynamics is much harder. And open systems where the Gibbs free-energy is more significant than simple entropy gradients are important to us, because it's part of the thermodynamic understanding of life.

Physicists play a sleight of hand, by defining information as the inverse of entropy. The full analysis of Maxwell's Demon links information-entropy and thermodynamics, so it makes sense even if in ordinary language information means a range of less specific things. Discussed here Is the concept of information nonphysical?

I see modern science as currently property-dualist, picturing everything as fundamentally constituted of energy, and information - which includes entropy. It was expected that information would resolve to being secondary properties of the fundamental things, mass-energy and spacetime. But now increasingly the reverse is expected, with those previously fundamental things pictured as emergent from information and it's propagation. Discussed here: Is information the foundation of reality?

We can understand 'true' as relating to comparing expectation and reality, and specifically in regards to building a tables of probabilities like the dice-rolling example, composed of counterfactuals. Discussed here: Why is a measured true value “TRUE”? Deutsch & Marletto's Constructor Theory looks to relate physics and information-theory in a deeper way, by expanding analysis of computation into sets of outcomes instead of a single floating-point variable, comparable to expanding our understanding of now to include Many Worlds.

Answer 2

a) Entropy is not about probabilities (although probabilities might be used to calculate it). In simple terms, entropy is a quantity that measures the potential of disorder. It is not a direct measure of disorder. The fact that low entropy implies high predictability is due to the reduction of the potential disorder, not to the reduction of disorder. At S=0, there is no potential for disorder, so, no disorder.

b) Entropy is a macrostatic quantity. Memorize that. That is: it is a descriptor of facts of perception, that is, ideas, sensations (for example, temperature is a macrostatic quantity: it is a feeling, which (thanks, 0th Law, sorry you didn't were defined first), it can be assessed as a physical quantity).

c) While macrostatic facts are related to ideas, microstatic facts are those related with physical phenomena. For example, microstatically, temperature is a quantity proportional to the statistical kinetic energy load of each molecule in a gas container.

d) Statistical entropy (S=k ln $ \omega $) describes the information carried for some macroscopic state:

S=ln2(64)=6

That is, in order to carry 64 states, 6 bits are necessary (logarithm base 2). So, in this example, the entropy of a system which can be in 64 states is 6.

d) Contrary to entropy, information is a microstatic quantity. Following the previous example, 111000 is the information that corresponds for a particular microstate of the system. Although this information is effectively carried in 6 bits, that is not necessarily the same of the entropy of the system. For example, Gibbs entropy can have a different value for the same 6 bits of information.

e) Thermodynamic entropy is equivalent to statistical entropy: it is a macrostatic quantity that measures the potential of disorder of a particular microstate. Perhaps the best interpretation here is to say that thermodynamic entropy is the measure of energetic order.

f) The big difference between thermodynamic and statistical entropy is that the latter allows measuring each microstate, at the cost of addressing systems always as discrete entities.

In addition to know that the entropy of a 6-bits message is 6, each message (the information) can also be obtained: 111000. Statistical entropy is mostly used to analyze fixed-size messages (but yes, it also allows addressing messages of different sizes). That is, statistical entropy is mostly used not to measure disorder, but how to assess the performance of a process at different values of disorder.

But the for the former, thermodynamic entropy, microstates cannot be measured. How would we know the actual configuration of X molecules having a value of entropy of S=0.nnnnn...? In addition, the absolute value of entropy of any substance cannot be measured directly, the only measurable value is dS=dQ/T, that is, change. So, thermodynamic entropy is mostly used to measure disorder as such.

Don't forget that S=0 is a convention for zero temperature, which experimentally shows that mass tends to order at 0∘K.

Answer 3

Information and thermodynamic entropies are in some way parallel concepts. Both describe how the complexity of the system increases over time. More and more information is needed to describe the state of the system.

When the signal to noise ratio decreases over time, the information in the signal does not disappear, it is only diluted in the constant influx of random noise. Random information is added to the signal.

Something like that is happening also in thermodynamic processes. Differences in energy density are parallel to the signal. Thermal noise, random vibration of particles distributes the energy more evenly.

It all boils down to the inherent randomness of a probabilistic universe. In a deterministic universe there would be no noise, complexity would not increase, all entropies would remain constant.