# What are the units of entropy of a normal distribution?

I have a random process that follows a normal distribution. It's parameters are mean = 35 units, std.dev. = 8 units. I've seen from the wiki entry for the normal distribution that there is a formula to calculate the entropy. So plugging in the figures as:- $$.5\log\left(2\pi e^1 8\cdot 8\right)$$ I get a value of 1.52, which I take to be per sample. My question is what are these units? What thing do I have 1.52 of?

Information entropy is (typically) measured in units of bits, after Claude Shannon's definition. So can I take it that each sample generates 1.52 bits of entropy? Clearly recording those samples generates information and therefore occupies a real and discrete amount of storage space. Ergo entropy cannot be unit less.

• entropy is unitless – Aksakal Sep 27 '17 at 2:12
• Yes, unless you meant something else by "unit" – Aksakal Sep 27 '17 at 3:26
• However the $\sigma$ may be in meters. Do we have something like $\log$ square meters? – Karel Macek Sep 27 '17 at 4:01
• You link to information entropy, which is shannon entropy, that is, the discrete case. You ask about entropy for the normal distribution, that is differential entropy, something entirely different. – kjetil b halvorsen Sep 27 '17 at 9:45
• The point is that shannon and differential entropy has very different properties and must be treated separately – kjetil b halvorsen Sep 27 '17 at 9:56

Some details: We treat Shannon (discrete) and differential (continuous) entropy separately. $$\DeclareMathOperator{\E}{\mathbb{E}} H(X) = -\sum_x p(x) \log p(x) = -\E_X \log p(X)$$ where $p$ is the probability mass function of a discrete random variable. Then $$H_d(X) = -\int f(x) \log f(x) \; dx = -\E_X \log f(X)$$ where $f$ is the probability density function of a continuous random variable. Now, from general principles the unit of measurement of the expectation (mean, average) of a variable (random or not) is the same as the unit of measurement of the variable itself. This leaves us with the unit of measurement of $\log p(x), \log f(x)$ respectively. Again, from general principles (see lognormal distribution, standard-deviation and (physical) units for discussion and references) the arguments of transcendental functions like $\log$ must be unitless. That rises a problem, while $p(x)$ certainly is unitless, since probability is an absolute number, the density $f(x)$ measures probability pr unit of $x$, so if unit of $x$ is $\text{u}$, then unit of $f(x)$ is $\text{u}^{-1}$. So, for the equation defining differential entropy $H_d$ to be dimensionally correct, we must assume the argument to log contains a "hidden" constant with numerical value 1 and unit $\text{u}$. But the conclusion follows, that both Shannon and differential entropy is unitless. Still, one must remember that differential entropy scales with the unit of measurement of $X$, as discussed in https://en.wikipedia.org/wiki/Differential_entropy