Variance of a bounded random variable

Question

Suppose that a random variable has a lower and an upper bound [0,1]. How to compute the variance of such a variable?

Answer 1

You can prove Popoviciu's inequality as follows. Use the notation $m=\inf X$ and $M=\sup X$. Define a function $g$ by $$ g(t)=\mathbb{E}\!\left[\left(X-t\right)^2\right] \, . $$ Computing the derivative $g'$, and solving $$ g'(t) = -2\mathbb{E}[X] +2t=0 \, , $$ we find that $g$ achieves its minimum at $t=\mathbb{E}[X]$ (note that $g''>0$).

Now, consider the value of the function $g$ at the special point $t=\frac{M+m}{2}$. It must be the case that $$ \mathbb{Var}[X]=g(\mathbb{E}[X])\leq g\left(\frac{M+m}{2}\right) \, . $$ But $$ g\left(\frac{M+m}{2}\right) = \mathbb{E}\!\left[\left(X - \frac{M+m}{2}\right)^2 \right] = \frac{1}{4}\mathbb{E}\!\left[\left((X-m) + (X-M)\right)^2 \right] \, . $$ Since $X-m\geq 0$ and $X-M\leq 0$, we have $$ \left((X-m)+(X-M)\right)^2\leq\left((X-m)-(X-M)\right)^2=\left(M-m\right)^2 \, , $$ implying that $$ \frac{1}{4}\mathbb{E}\!\left[\left((X-m) + (X-M)\right)^2 \right] \leq \frac{1}{4}\mathbb{E}\!\left[\left((X-m) - (X-M)\right)^2 \right] = \frac{(M-m)^2}{4} \, . $$ Therefore, we proved Popoviciu's inequality $$ \mathbb{Var}[X]\leq \frac{(M-m)^2}{4} \, . $$

Answer 2

If the random variable is restricted to $[a,b]$ and we know the mean $\mu=E[X]$, the variance is bounded by $(b-\mu)(\mu-a)$.

Let us first consider the case $a=0, b=1$. Note that for all $x\in [0,1]$, $x^2\leq x$, wherefore also $E[X^2]\leq E[X]$. Using this result, \begin{equation} \sigma^2 = E[X^2] - (E[X]^2) = E[X^2] - \mu^2 \leq \mu - \mu^2 = \mu(1-\mu). \end{equation}

To generalize to intervals $[a,b]$ with $b>a$, consider $Y$ restricted to $[a,b]$. Define $X=\frac{Y-a}{b-a}$, which is restricted in $[0,1]$. Equivalently, $Y = (b-a)X + a$, and thus \begin{equation} Var[Y] = (b-a)^2Var[X] \leq (b-a)^2\mu_X (1-\mu_X). \end{equation} where the inequality is based on the first result. Now, by substituting $\mu_X = \frac{\mu_Y - a}{b-a}$, the bound equals \begin{equation} (b-a)^2\, \frac{\mu_Y - a}{b-a}\,\left(1- \frac{\mu_Y - a}{b-a}\right) = (b-a)^2 \frac{\mu_Y -a}{b-a}\,\frac{b - \mu_Y}{b-a} = (\mu_Y - a)(b- \mu_Y), \end{equation} which is the desired result.

Answer 3

Let $F$ be a distribution on $[0,1]$. We will show that if the variance of $F$ is maximal, then $F$ can have no support in the interior, from which it follows that $F$ is Bernoulli and the rest is trivial.

As a matter of notation, let $\mu_k = \int_0^1 x^k dF(x)$ be the $k$th raw moment of $F$ (and, as usual, we write $\mu = \mu_1$ and $\sigma^2 = \mu_2 - \mu^2$ for the variance).

We know $F$ does not have all its support at one point (the variance is minimal in that case). Among other things, this implies $\mu$ lies strictly between $0$ and $1$. In order to argue by contradiction, suppose there is some measurable subset $I$ in the interior $(0,1)$ for which $F(I)\gt 0$. Without any loss of generality we may assume (by changing $X$ to $1-X$ if need be) that $F(J = I \cap (0, \mu]) \gt 0$: in other words, $J$ is obtained by cutting off any part of $I$ above the mean and $J$ has positive probability.

Let us alter $F$ to $F'$ by taking all the probability out of $J$ and placing it at $0$. In so doing, $\mu_k$ changes to

$$\mu'_k = \mu_k - \int_J x^k dF(x).$$

As a matter of notation, let us write $[g(x)] = \int_J g(x) dF(x)$ for such integrals, whence

$$\mu'_2 = \mu_2 - [x^2], \quad \mu' = \mu - [x].$$

Calculate

$$\sigma'^2 = \mu'_2 - \mu'^2 = \mu_2 - [x^2] - (\mu - [x])^2 = \sigma^2 + \left((\mu[x] - [x^2]) + (\mu[x] - [x]^2)\right).$$

The second term on the right, $(\mu[x] - [x]^2)$, is non-negative because $\mu \ge x$ everywhere on $J$. The first term on the right can be rewritten

$$\mu[x] - [x^2] = \mu(1 - [1]) + ([\mu][x] - [x^2]).$$

The first term on the right is strictly positive because (a) $\mu \gt 0$ and (b) $[1] = F(J) \lt 1$ because we assumed $F$ is not concentrated at a point. The second term is non-negative because it can be rewritten as $[(\mu-x)(x)]$ and this integrand is nonnegative from the assumptions $\mu \ge x$ on $J$ and $0 \le x \le 1$. It follows that $\sigma'^2 - \sigma^2 \gt 0$.

We have just shown that under our assumptions, changing $F$ to $F'$ strictly increases its variance. The only way this cannot happen, then, is when all the probability of $F'$ is concentrated at the endpoints $0$ and $1$, with (say) values $1-p$ and $p$, respectively. Its variance is easily calculated to equal $p(1-p)$ which is maximal when $p=1/2$ and equals $1/4$ there.

Now when $F$ is a distribution on $[a,b]$, we recenter and rescale it to a distribution on $[0,1]$. The recentering does not change the variance whereas the rescaling divides it by $(b-a)^2$. Thus an $F$ with maximal variance on $[a,b]$ corresponds to the distribution with maximal variance on $[0,1]$: it therefore is a Bernoulli$(1/2)$ distribution rescaled and translated to $[a,b]$ having variance $(b-a)^2/4$, QED.

Answer 4

At @user603's request....

A useful upper bound on the variance $\sigma^2$ of a random variable that takes on values in $[a,b]$ with probability $1$ is $\sigma^2 \leq \frac{(b−a)^2}{4}$. A proof for the special case $a=0, b=1$ (which is what the OP asked about) can be found here on math.SE, and it is easily adapted to the more general case. As noted in my comment above and also in the answer referenced herein, a discrete random variable that takes on values $a$ and $b$ with equal probability $\frac{1}{2}$ has variance $\frac{(b−a)^2}{4}$ and thus no tighter general bound can be found.

Another point to keep in mind is that a bounded random variable has finite variance, whereas for an unbounded random variable, the variance might not be finite, and in some cases might not even be definable. For example, the mean cannot be defined for Cauchy random variables, and so one cannot define the variance (as the expectation of the squared deviation from the mean).

Answer 5

Here's a really simple proof I found in Sheldon Ross's A first course in probability 10th ed., "theory problem 5.8".

Suppose we have a random variable $X$ between $0$ and $c$. Then $X \leq c$ and thus $E(X^2) \leq c E(X)$. We thus have

$$\mathrm{Var}(X) \leq c E(X) - E(X)^2 = c^2 \alpha (1 - \alpha) \leq c^2 / 4$$

where $\alpha \equiv E(X) / c$ and the final step follows from straightforward differentiation to see that $\alpha (1 - \alpha)$ is minimized at $\alpha = 1/2$.

Answer 6

Given random variable $D$ with mean $E D=\mu$, when $a\le D\le b$, we have \begin{eqnarray*} E (D-\mu)^2&\le& E[(D-\mu)^2-(D-a)(D-b)]\\ &=& E[\mu^2-2\mu D+(a+b)D-ab]\\ &=& (a+b)\mu-\mu^2-ab\\ &=&(\mu-a)(b-\mu), \end{eqnarray*} where the equation holds if and only if $D\in\{a,b\}$ with probability $1$.

Moreover, if $\mu$ can be arbitrary in the support set $[a,b]$, then clearly $$E(D-\mu)^2\le \left(\tfrac{a+b}{2}-a\right)\left(\tfrac{a+b}{2}-b\right)=\left(\tfrac{b-a}{2}\right)^2.$$

Answer 7

Wait, there is a two liner. Without loss of generality $\mathbb{E}X = 0, X \in [a, b]$. Then $$\left|X - \frac{b + a}{2}\right| \leq \frac{b - a}{2}.$$ Now $${\rm Var}(X) = {\rm Var}\left(X - \frac{b + a}{2}\right) \leq \mathbb{E}\left(X - \frac{b + a}{2}\right)^2 \leq \frac{(b - a)^2}{4}.$$

Edit: As pointed out by whuber, the second inequality needs a proof: ${\rm Var}(Y) = \mathbb{E}[Y^2] - \mathbb{E}[Y]^2 \leq \mathbb{E}[Y^2]$.

Answer 8

are you sure that this is true in general - for continuous as well as discrete distributions? Can you provide a link to the other pages? For a general distibution on $[a,b]$ it is trivial to show that $$ Var(X) = E[(X-E[X])^2] \le E[(b-a)^2] = (b-a)^2. $$ I can imagine that sharper inequalities exist ... Do you need the factor $1/4$ for your result?

On the other hand one can find it with the factor $1/4$ under the name Popoviciu's_inequality on wikipedia.

This article looks better than the wikipedia article ...

For a uniform distribution it holds that $$ Var(X) = \frac{(b-a)^2}{12}. $$

Answer 9

The key elements here are that $f(x) = x^2$ is convex, $EX$ minimises $E(X-t)^2$ and $X(\omega) \in [a,b]$.

Let $x \in [a,b]$, then $f(x-{1 \over 2}(a+b)) \le {1 \over 2} (f({x-a \over 2}) + f({x-b \over 2}))$, or $(x-{1 \over 2}(a+b))^2 \le {1 \over 2} (({x-a \over 2})^2 + ({x-b \over 2})^2 ) \le ({b-a \over 2})^2$.

Since $E(X-t)^2 = E(X-EX + EX-t)^2 = \operatorname{var} X + (EX-t)^2$ for any $t$, setting $t={1 \over 2}(a+b)$ shows that $ \operatorname{var} X \le E(X-{1 \over 2}(a+b))^2 \le ({b-a \over 2})^2$.

Answer 10

A simple proof of Popoviciu's inequality is the following, where $X\in [m, M]$: \begin{equation} \color{blue}{\operatorname{Var}[X] \le \operatorname{Var}[X] + \mathbb{E}[(M-X)(X-m)] = \frac{(M-m)^2}{4} - \left(\mathbb{E}[X] - \frac{M+m}{2}\right)^2 \le \frac{(M-m)^2}{4}.} \end{equation} Source: https://math.stackexchange.com/a/4264325/31498

(Check this answer out for a one-liner proof of Bhatia—Davis's inequality as well.)