21
$\begingroup$

For a uniformly distributed variable between 0 and 1 generated using

rand(1,10000)

this returns 10,000 random numbers between 0 and 1. If you take the mean, it is 0.5, while if you take the log of that sample, then take the mean of the result:

mean(log(rand(1,10000)))

I would expect that the result to be $\log 0.5=-.6931$, but instead the answer is -1. Why is this so?

$\endgroup$
  • 4
    $\begingroup$ A minute point ignored by either answer to date is that log 0 is indeterminate. I don't know whether MATLAB regards the distribution as having support (0, 1] or [0, 1) but this should be documented somewhere. Otherwise put, in principle your transformed distribution has an infinite left tail. $\endgroup$ – Nick Cox Oct 27 '19 at 8:19
  • 2
    $\begingroup$ @NickCox: apparently it does not produce zeros or ones. $\endgroup$ – Xi'an Oct 27 '19 at 18:48
  • 2
    $\begingroup$ Because the log of the mean isnt the same thing as the mean of the logs. $\endgroup$ – Marquis of Lorne Oct 27 '19 at 23:58
  • $\begingroup$ @Xi'an Thanks for the link. So,MATLAB uses a support of $(0, 1)$ which certainly avoids some very occasional problems. But as this question might interest others too, check out your software if different. $\endgroup$ – Nick Cox Oct 28 '19 at 14:36
  • 3
    $\begingroup$ Why would you think it should be so? Consider a uniform distribution between -1 and 1. E[x]=0. Then consider y=abs(x). abs(E[x])=0 but obviously E[abs(x)]>0. $\endgroup$ – MooseBoys Oct 28 '19 at 18:40
52
$\begingroup$

Consider two values symmetrically placed around $0.5$ - like $0.4$ and $0.6$ or $0.25$ and $0.75$. Their logs are not symmetric around $\log(0.5)$. $\log(0.5-\epsilon)$ is further from $\log(0.5)$ than $\log(0.5+\epsilon)$ is. So when you average them you get something less than $\log(0.5)$.

Similarly, if you take a teeny interval around a collection of such pairs of symmetrically placed values, you still get the average of the logs of each pair being below $\log(0.5)$... and it's a simple matter to move from that observation to the definition of the expectation of the log.

Indeed, usually, $E(t(X))\neq t(E(X))$ unless $t$ is linear.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Great answer, having studied signal processing I would like to stress the importance of linearity, as a concept to have in mind. The last sentence is perfect in itself, but as you have a very "easy" (and good) explanation in the first two paragraphs some people might be at a loss in the third. And as it is the most important to my mind, I feel elaborating it a bit would be great. $\endgroup$ – Goodbye SE Oct 29 '19 at 7:55
53
$\begingroup$

This is another illustration of Jensen's inequality $$\mathbb E[\log X] < \log \mathbb E[X]$$ (since the function $x\mapsto \log(x)$ is strictly concave] and of the more general (anti-)property that the expectation of the transform is not the transform of the expectation when the transform is not linear (plus a few exotic cases). (Most of my undergraduate students are however firm believers in the magical identity $\mathbb E[h(X)] = h(\mathbb E[X])$ if I only judge from the frequency of this equality appearing in their final exam papers.)

| cite | improve this answer | |
$\endgroup$
  • 4
    $\begingroup$ +1 for mentioning concave. An illustration with a curve might make the point even clearer. $\endgroup$ – Eric Duminil Oct 27 '19 at 13:14
8
$\begingroup$

It is worthwhile to note that if $X \sim \operatorname{Uniform}(0,1)$, then $-\log X \sim \operatorname{Exponential}(\lambda = 1)$, so that $\operatorname{E}[\log X] = -1$. Explicitly, $$f_X(x) = \mathbb 1(0 < x < 1) = \begin{cases} 1, & 0 < x < 1 \\ 0, & \text{otherwise} \end{cases}$$ implies $$Y = g(X) = -\log X$$ has density $$\begin{align*} f_Y(y) &= f_X(g^{-1}(y)) \left|\frac{dg^{-1}}{dy}\right| \\ &= \mathbb 1 \left( 0 < e^{-y} < 1 \right) \left| - e^{-y} \right| \\ &= e^{-y} \mathbb 1 (0 < y < \infty) \\ &= \begin{cases} e^{-y}, & y > 0 \\ 0, & \text{otherwise}. \end{cases} \end{align*}$$ Thus $Y \sim \operatorname{Exponential}(\lambda = 1)$ and its mean is $1$. This furnishes a very convenient method to generate exponentially distributed random variables via log-transformation of a uniform random variable on $(0,1)$.

| cite | improve this answer | |
$\endgroup$
6
$\begingroup$

Note that the mean of a transformed uniform variable is just the mean value of the function doing the transformation over the domain (since we are expecting each value to be selected equally). This is simply,

$$ \frac{1}{b-a}\int_a^b{t(x)}dx = \int_0^1{t(x)}dx $$

For example (in R):

$$ \int_0^1{log(x)}dx = (1\cdot log(1)-1) - 0 = 0-1 =-1 $$

mean(log(runif(1e6)))
[1] -1.000016
integrate(function(x) log(x), 0, 1)
-1 with absolute error < 1.1e-15

$$ \int_0^1{x^2}dx = \frac{1}{3}(1^3-0^3) = \frac{1}{3} $$

mean(runif(1e6)^2)
[1] 0.3334427
integrate(function(x) (x)^2, 0, 1)
0.3333333 with absolute error < 3.7e-15

$$ \int_0^1{e^x}dx = e^1-e^0 = e-1 $$

mean(exp(runif(1e6)))
[1] 1.718425
integrate(function(x) exp(x), 0, 1)
1.718282 with absolute error < 1.9e-14
exp(1)-1
[1] 1.718282
| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.