Difference between different kinds of entropy

Question

The following concepts are baffling and would be obliged for a constructive explanation.

(Q1) What is the conceptual difference between (a) Kolmogorov-Sinai entropy, (b) Shannon entropy, (c) Source entropy (d) topological entropy and (e) Boltzmann entropy?

(Q2) What are the relationships between different entropies?

(Q3) Are they interchangeable and mean the same?

(Q4) Where can I find a good lucid explanation about Kolmogorov entropy and why it is needed and its significance?

(Q5) Can Boltzmann entropy be equal to Kolmogorov entropy?

Answer 1

Shannon entropy is a general concept for entropy for any probability distribution. And depending what probability you choose, you get something different.

In particular:

• Source entropy is Shannon entropy of probability distribution associated with some source of signals - for example zeros and ones or letter over an alphabet (see C. Shannon, A Mathematical Theory of Communication (1948))
• Boltzmann entropy is Shannon entropy of the probability distribution of statistical microstates (same: of probability distribution in the phase space, up to a constant factor)

And about entropies I am not deep into:

• Kolmogorov-Sinai Entropy, is (as far I understand) a particular quantity derived from entropy, related to chaotic behavior of dynamic systems (it also happens in the phase space); as you see its formula, its formula involves Shannon entropy for a particular set related to trajectories, but also involves other operations
• Topological entropy is a related concept