definition

Information and entropy: opposite for physicists vs Shannon

Computer scientists and physicists give opposite answers about the relation between entropy and information: in Shannon's communication theory, high entropy (uniform probability distribution) means high information because each symbol is maximally surprising, whereas in Boltzmann's statistical mechanics, low entropy means high information because specifying a low-entropy macrostate pins down the microstate far more tightly.

definitionpending

Speaker

Sean Carroll

Evidence Quote

“the communication theorists or information theorists think that information and entropy are in the same direction. Physicists, statistical mechanics physicists anyway, thinking in Boltzmann's way, think that information and entropy are opposite to each other.”

Source

320 | Solo: Complexity and the Universe— Sean Carroll's Mindscape
Created: 6/13/2026, 12:13:02 AM

Your notes

Loading notes...