Suppose that a random variable has a lower and an upper bound [0,1]. How to compute the variance of such a variable?

  • 7
    The same way as for an unbounded variable - setting integration or summation limits appropriately. – Scortchi Dec 10 '12 at 13:46
  • 2
    As @Scortchi said. But I'm curious why you thought it might be different? – Peter Flom Dec 10 '12 at 14:09
  • 3
    Unless you don't know anything about the variable (in which case an upper bound on variance might be calculated from the existence of bounds), why would the fact that it's bounded come into the calculation? – Glen_b Dec 10 '12 at 22:15
  • 5
    A useful upper bound on the variance of a random variable that takes on values in $[a,b]$ with probability $1$ is $(b-a)^2/4$ and is achieved by a discrete random variable that takes on values $a$ and $b$ with equal probability $\frac{1}{2}$. Another point to keep in mind is that the variance is guaranteed to exist whereas an unbounded random variable might not have a variance (some, such as Cauchy random variables don't even have a mean). – Dilip Sarwate Dec 11 '12 at 17:44
  • 7
    There is a discrete random variable whose variance equals $\frac{(b-a)^2}{4}$ exactly: a random variable that takes on values $a$ and $b$ with equal probability $\frac{1}{2}$. So, at least we know that a universal upper bound on the variance cannot be smaller than $\frac{(b-a)^2}{4}$. – Dilip Sarwate Feb 21 '13 at 18:01

You can prove Popoviciu's inequality as follows. Use the notation $m=\inf X$ and $M=\sup X$. Define a function $g$ by $$ g(t)=\mathbb{E}\left[\left(X-t\right)^2\right] \, . $$ Computing the derivative $g'$, and solving $$ g'(t) = -2\mathbb{E}[X] +2t=0 \, , $$ we find that $g$ achieves its minimum at $t=\mathbb{E}[X]$ (note that $g''>0$).

(Actually, you don't need to compute this derivative; see Dilip's "old school way" in his comment bellow.)

Now, consider the value of the function $g$ at the special point $t=\frac{M+m}{2}$. It must be the case that $$ \mathbb{Var}[X]=g(\mathbb{E}[X])\leq g\left(\frac{M+m}{2}\right) \, . $$ But $$ g\left(\frac{M+m}{2}\right) = \mathbb{E}\left[\left(X - \frac{M+m}{2}\right)^2 \right] = \frac{1}{4}\mathbb{E}\left[\left((X-m) + (X-M)\right)^2 \right] \, . $$ Since $X-m\geq 0$ and $X-M\leq 0$, we have $$ \left((X-m)+(X-M)\right)^2\leq\left((X-m)-(X-M)\right)^2=\left(M-m\right)^2 \, , $$ implying that $$ \frac{1}{4}\mathbb{E}\left[\left((X-m) + (X-M)\right)^2 \right] \leq \frac{1}{4}\mathbb{E}\left[\left((X-m) - (X-M)\right)^2 \right] = \frac{(M-m)^2}{4} \, . $$ Therefore, we proved Popoviciu's inequality $$ \mathbb{Var}[X]\leq \frac{(M-m)^2}{4} \, . $$

  • 3
    Nice approach: it's good to see rigorous demonstrations of these kinds of things. – whuber Feb 21 '13 at 18:00
  • 17
    +1 Nice! I learned statistics long before computers were in vogue, and one idea that was drilled into us was that $$E[(X-t)^2] = E[((X-\mu)-(t-\mu))^2] = E[(X-\mu)^2]+(t-\mu)^2$$ which allowed for the computation of variance by finding the sum of the squares of the deviations from any convenient point $t$ and then adjusting for the bias. Here of course, this identity gives a simple proof of the result that $g(t)$ has minimum value at $t=\mu$ without the necessity of derivatives etc. – Dilip Sarwate Feb 21 '13 at 22:22

Let $F$ be a distribution on $[0,1]$. We will show that if the variance of $F$ is maximal, then $F$ can have no support in the interior, from which it follows that $F$ is Bernoulli and the rest is trivial.

As a matter of notation, let $\mu_k = \int_0^1 x^k dF(x)$ be the $k$th raw moment of $F$ (and, as usual, we write $\mu = \mu_1$ and $\sigma^2 = \mu_2 - \mu^2$ for the variance).

We know $F$ does not have all its support at one point (the variance is minimal in that case). Among other things, this implies $\mu$ lies strictly between $0$ and $1$. In order to argue by contradiction, suppose there is some measurable subset $I$ in the interior $(0,1)$ for which $F(I)\gt 0$. Without any loss of generality we may assume (by changing $X$ to $1-X$ if need be) that $F(J = I \cap (0, \mu]) \gt 0$: in other words, $J$ is obtained by cutting off any part of $I$ above the mean and $J$ has positive probability.

Let us alter $F$ to $F'$ by taking all the probability out of $J$ and placing it at $0$. In so doing, $\mu_k$ changes to

$$\mu'_k = \mu_k - \int_J x^k dF(x).$$

As a matter of notation, let us write $[g(x)] = \int_J g(x) dF(x)$ for such integrals, whence

$$\mu'_2 = \mu_2 - [x^2], \quad \mu' = \mu - [x].$$

Calculate

$$\sigma'^2 = \mu'_2 - \mu'^2 = \mu_2 - [x^2] - (\mu - [x])^2 = \sigma^2 + \left((\mu[x] - [x^2]) + (\mu[x] - [x]^2)\right).$$

The second term on the right, $(\mu[x] - [x]^2)$, is non-negative because $\mu \ge x$ everywhere on $J$. The first term on the right can be rewritten

$$\mu[x] - [x^2] = \mu(1 - [1]) + ([\mu][x] - [x^2]).$$

The first term on the right is strictly positive because (a) $\mu \gt 0$ and (b) $[1] = F(J) \lt 1$ because we assumed $F$ is not concentrated at a point. The second term is non-negative because it can be rewritten as $[(\mu-x)(x)]$ and this integrand is nonnegative from the assumptions $\mu \ge x$ on $J$ and $0 \le x \le 1$. It follows that $\sigma'^2 - \sigma^2 \gt 0$.

We have just shown that under our assumptions, changing $F$ to $F'$ strictly increases its variance. The only way this cannot happen, then, is when all the probability of $F'$ is concentrated at the endpoints $0$ and $1$, with (say) values $1-p$ and $p$, respectively. Its variance is easily calculated to equal $p(1-p)$ which is maximal when $p=1/2$ and equals $1/4$ there.

Now when $F$ is a distribution on $[a,b]$, we recenter and rescale it to a distribution on $[0,1]$. The recentering does not change the variance whereas the rescaling divides it by $(b-a)^2$. Thus an $F$ with maximal variance on $[a,b]$ corresponds to the distribution with maximal variance on $[0,1]$: it therefore is a Bernoulli$(1/2)$ distribution rescaled and translated to $[a,b]$ having variance $(b-a)^2/4$, QED.

  • Interesting, whuber. I didn't know this proof. – Zen Feb 21 '13 at 18:01
  • 6
    @Zen It's by no means as elegant as yours. I offered it because I have found myself over the years thinking in this way when confronted with much more complicated distributional inequalities: I ask how the probability can be shifted around in order to make the inequality more extreme. As an intuitive heuristic it's useful. By using approaches like the one laid out here, I suspect a general theory for proving a large class of such inequalities could be derived, with a kind of hybrid flavor of the Calculus of Variations and (finite dimensional) Lagrange multiplier techniques. – whuber Feb 21 '13 at 18:04
  • Perfect: your answer is important because it describes a more general technique that can be used to handle many other cases. – Zen Feb 21 '13 at 18:06
  • @whuber said - "I ask how the probability can be shifted around in order to make the inequality more extreme." -- this seems to be the natural way to think about such problems. – Glen_b Feb 21 '13 at 23:40
  • There appear to be a few mistakes in the derivation. It should be $$\mu[x] - [x^2] = \mu(1 - [1])[x] + ([\mu][x] - [x^2]).$$ Also, $[(\mu-x)(x)]$ does not equal $[\mu][x] - [x^2]$ since $[\mu][x]$ is not the same as $\mu[x]$ – Leo Aug 8 '13 at 12:12

If the random variable is restricted to $[a,b]$ and we know the mean $\mu=E[X]$, the variance is bounded by $(b-\mu)(\mu-a)$.

Let us first consider the case $a=0, b=1$. Note that for all $x\in [0,1]$, $x^2\leq x$, wherefore also $E[X^2]\leq E[X]$. Using this result, \begin{equation} \sigma^2 = E[X^2] - (E[X]^2) = E[X^2] - \mu^2 \leq \mu - \mu^2 = \mu(1-\mu). \end{equation}

To generalize to intervals $[a,b]$ with $b>a$, consider $Y$ restricted to $[a,b]$. Define $X=\frac{Y-a}{b-a}$, which is restricted in $[0,1]$. Equivalently, $Y = (b-a)X + a$, and thus \begin{equation} Var[Y] = (b-a)^2Var[X] \leq (b-a)^2\mu_X (1-\mu_X). \end{equation} where the inequality is based on the first result. Now, by substituting $\mu_X = \frac{\mu_Y - a}{b-a}$, the bound equals \begin{equation} (b-a)^2\, \frac{\mu_Y - a}{b-a}\,\left(1- \frac{\mu_Y - a}{b-a}\right) = (b-a)^2 \frac{\mu_Y -a}{b-a}\,\frac{b - \mu_Y}{b-a} = (\mu_Y - a)(b- \mu_Y), \end{equation} which is the desired result.

At @user603's request....

A useful upper bound on the variance $\sigma^2$ of a random variable that takes on values in $[a,b]$ with probability $1$ is $\sigma^2 \leq \frac{(b−a)^2}{4}$. A proof for the special case $a=0, b=1$ (which is what the OP asked about) can be found here on math.SE, and it is easily adapted to the more general case. As noted in my comment above and also in the answer referenced herein, a discrete random variable that takes on values $a$ and $b$ with equal probability $\frac{1}{2}$ has variance $\frac{(b−a)^2}{4}$ and thus no tighter general bound can be found.

Another point to keep in mind is that a bounded random variable has finite variance, whereas for an unbounded random variable, the variance might not be finite, and in some cases might not even be definable. For example, the mean cannot be defined for Cauchy random variables, and so one cannot define the variance (as the expectation of the squared deviation from the mean).

  • this is a special case of @Juho's answer – Aksakal Apr 11 '14 at 20:00
  • It was just a comment, but I could also add that this answer does not answer the question asked. – Aksakal Apr 11 '14 at 20:05
  • @Aksakal So??? Juho was answering a slightly different and much more recently asked question. This new question has been merged with the one you see above, which I answered ten months ago. – Dilip Sarwate Apr 11 '14 at 20:07

are you sure that this is true in general - for continuous as well as discrete distributions? Can you provide a link to the other pages? For a general distibution on $[a,b]$ it is trivial to show that $$ Var(X) = E[(X-E[X])^2] \le E[(b-a)^2] = (b-a)^2. $$ I can imagine that sharper inequalities exist ... Do you need the factor $1/4$ for your result?

On the other hand one can find it with the factor $1/4$ under the name Popoviciu's_inequality on wikipedia.

This article looks better than the wikipedia article ...

For a uniform distribution it holds that $$ Var(X) = \frac{(b-a)^2}{12}. $$

  • This page states the result with the start of a proof that gets a bit too involved for me as it seems to require an understanding of the "Fundamental Theorem of Linear Programming". sci.tech-archive.net/Archive/sci.math/2008-06/msg01239.html – Adam Russell Feb 21 '13 at 17:14
  • Thank you for putting a name to this! "Popoviciu's Inequality" is just what I needed. – Adam Russell Feb 21 '13 at 17:41
  • 2
    This answer makes some incorrect suggestions: $1/4$ is indeed right. The reference to Popoviciu's inequality will work, but strictly speaking it applies only to distributions with finite support (in particular, that includes no continuous distributions). A limiting argument would do the trick, but something extra is needed here. – whuber Feb 21 '13 at 17:58
  • 2
    A continuous distribution can approach a discrete one (in cdf terms) arbitrarily closely (e.g. construct a continuous density from a given discrete one by placing a little Beta(4,4)-shaped kernel centered at each mass point - of the appropriate area - and let the standard deviation of each such kernel shrink toward zero while keeping its area constant). Such discrete bounds as discussed here will thereby also act as bounds on continuous distributions. I expect you're thinking about continuous unimodal distributions... which indeed have different upper bounds. – Glen_b Feb 21 '13 at 23:44
  • 2
    Well ... my answer was the least helpful but I would leave it here due to the nice comments. Cheers,R – Richard Feb 22 '13 at 8:35

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.