# What are the units of entropy of a normal distribution?

I have a random process that follows a normal distribution. It's parameters are mean = 35 units, std.dev. = 8 units. I've seen from the wiki entry for the normal distribution that there is a formula to calculate the entropy. So plugging in the figures as:- $$.5\log\left(2\pi e^1 8\cdot 8\right)$$ I get a value of 1.52, which I take to be per sample. My question is what are these units? What thing do I have 1.52 of?

Information entropy is (typically) measured in units of bits, after Claude Shannon's definition. So can I take it that each sample generates 1.52 bits of entropy? Clearly recording those samples generates information and therefore occupies a real and discrete amount of storage space. Ergo entropy cannot be unit less.

• entropy is unitless – Aksakal Sep 27 '17 at 2:12
• Yes, unless you meant something else by "unit" – Aksakal Sep 27 '17 at 3:26
• However the $\sigma$ may be in meters. Do we have something like $\log$ square meters? – Karel Macek Sep 27 '17 at 4:01
• You link to information entropy, which is shannon entropy, that is, the discrete case. You ask about entropy for the normal distribution, that is differential entropy, something entirely different. – kjetil b halvorsen Sep 27 '17 at 9:45
• The point is that shannon and differential entropy has very different properties and must be treated separately – kjetil b halvorsen Sep 27 '17 at 9:56

Some details: We treat Shannon (discrete) and differential (continuous) entropy separately. $$\DeclareMathOperator{\E}{\mathbb{E}} H(X) = -\sum_x p(x) \log p(x) = -\E_X \log p(X)$$ where $$p$$ is the probability mass function of a discrete random variable. Then $$H_d(X) = -\int f(x) \log f(x) \; dx = -\E_X \log f(X)$$ where $$f$$ is the probability density function of a continuous random variable. Now, from general principles the unit of measurement of the expectation (mean, average) of a variable (random or not) is the same as the unit of measurement of the variable itself. This leaves us with the unit of measurement of $$\log p(x), \log f(x)$$ respectively. Again, from general principles (see lognormal distribution, standard-deviation and (physical) units for discussion and references) the arguments of transcendental functions like $$\log$$ must be unitless. That rises a problem, while $$p(x)$$ certainly is unitless, since probability is an absolute number, the density $$f(x)$$ measures probability pr unit of $$x$$, so if unit of $$x$$ is $$\text{u}$$, then unit of $$f(x)$$ is $$\text{u}^{-1}$$. So, for the equation defining differential entropy $$H_d$$ to be dimensionally correct, we must assume the argument to log contains a "hidden" constant with numerical value 1 and unit $$\text{u}$$. But the conclusion follows, that both Shannon and differential entropy is unitless. Still, one must remember that differential entropy scales with the unit of measurement of $$X$$, as discussed in https://en.wikipedia.org/wiki/Differential_entropy