9
$\begingroup$

For a given constant number $r$ (e.g. 4), is it possible to find a probability distribution for $X$, so that we have $\mathrm{Var}(X)=r$?

$\endgroup$
  • 1
    $\begingroup$ No, unless you've extra info. $\endgroup$ – Hemant Rupani Jul 2 '15 at 9:31
  • $\begingroup$ @Hemant Rupani what extra information is needed? $\endgroup$ – Amir Ef Jul 2 '15 at 9:32
  • 1
    $\begingroup$ any Nature of Random Variable 'X'… $\endgroup$ – Hemant Rupani Jul 2 '15 at 9:35
  • 3
    $\begingroup$ I suggest you edit your question to replace "value for X" with "distribution for X" - if X has just a single value, then X has a degenerate distribution and will have variance zero. $\endgroup$ – Silverfish Jul 2 '15 at 9:42
  • 1
    $\begingroup$ Unless $r$ is negative the answer is obviously yes, a variance can be any positive number. $\endgroup$ – dsaxton Jul 2 '15 at 14:07
13
$\begingroup$

Carefully considering the cases for $r$: if $r=0$ then the distribution is degenerate, but $X$ could have any mean. That is, $\Pr(X=\mu)=1$ and $\Pr(X=c)=0$ for any $c \neq \mu$. So we can find many possible distributions for $X$, but they are indexed by, and completely specified by, $\mu \in \mathbb{R}$.

If $r<0$ , no distribution can be found, since $\mathrm{Var}(X)=\mathbb{E}(X-\mu_X)^2 \geq 0$.

For $r>0$, the answer will depend on what additional information is known about $X$. For example if $X$ is known to have mean $\mu$, then for any $\mu \in \mathbb{R}$ and $r>0$ we can find a distribution with these moments by taking $X \sim N(\mu, r)$. This is not a unique solution to the problem of matching mean and variance, but it is the only normally distributed solution (and of all the possible solutions, this is the one that maximises the entropy, as Daniel points out). If you also wanted to match e.g. the third central moment, or higher, then you would need to consider a broader range of probability distributions.

Suppose instead we had some information about the distribution of $X$ rather than its moments. For example, if we know that $X$ follows a Poisson distribution then the unique solution would be $X \sim \mathrm{Poisson}(r)$. If we know that $X$ follows an exponential distribution, then again there is a unique solution $X \sim \mathrm{Exponential}(\frac{1}{\sqrt{r}})$, where we have found the parameter by solving $\mathrm{Var}(X) = r = \frac{1}{\lambda^2}$.

In other cases we can find an entire family of solutions. If we know that $X$ follows a rectangular (continuous uniform) distribution, then we can find a unique width $w$ for the distribution by solving $\mathrm{Var}(X) = r = \frac{w^2}{12}$. But there will be a whole family of solutions, $X \sim U(a, a+w)$ parametized by $a \in \mathbb{R}$ — the distributions in this set are all translations of each other. Similarly, if $X$ is normal then any distribution $X \sim N(\mu, r)$ would work (so we have a whole set of solutions indexed by $\mu$, which again can be any real number, and again the family are all translations of each other). If $X$ follows a gamma distribution then, using the shape-scale parameterization, we can obtain a whole family of solutions, $X \sim \mathrm{Gamma}(\frac{r}{\theta^2}, \theta)$ parametized by $\theta > 0$. Members of this family are not translations of each other. To help visualize what a "family of solutions" might look like, here are some examples of normal distributions indexed by $\mu$, and then gamma distributions indexed by $\theta$, all with variance equal to four, corresponding to the example $r=4$ in your question.

Normal distributions with variance four Gamma distributions with variance four

On the other hand, for some distributions it may or may not be possible to find a solution, depending on the value of $r$. For instance if $X$ must be a Bernoulli variable then for $0 \leq r \lt 0.25$ there are two possible solutions $X \sim \mathrm{Bernoulli}(p)$ because there are two probabilities $p$ which solve the equation $\mathrm{Var}(X) = r = p(1-p)$, and in fact these two probabilities are complementary i.e. $p_1 + p_2 = 1$. For $r=0.25$ there is only the unique solution $p=0.5$, and for $r>0.25$ no Bernoulli distribution has sufficiently high variance.

I feel I should also mention the case $r = \infty$. There are solutions for this case too, for example a Student's $t$ distribution with two degrees of freedom.

R code for plots

require(ggplot2)

x.df  <- data.frame(x = rep(seq(from=-8, to=8, length=100), times=5),
    mu = rep(c(-4, -2, 0, 2, 4), each=100))
x.df$pdf <- dnorm(mean=x.df$mu, x.df$x)
ggplot(x.df, aes(x=x, y=pdf, group=factor(mu), colour=factor(mu))) + theme_bw() + 
    geom_line(size=1) + scale_colour_brewer(name=expression(mu), palette="Set1") +
    theme(legend.key = element_blank()) + ggtitle("Normal distributions with variance 4")

x.df  <- data.frame(x = rep(seq(from=0, to=20, length=1000), times=5),
    theta = rep(c(0.25, 0.5, 1, 2, 4), each=1000))
x.df$pdf <- dgamma(x.df$x, shape=4/(x.df$theta)^2, scale=x.df$theta)
ggplot(x.df, aes(x=x, y=pdf, group=factor(theta), colour=factor(theta))) + theme_bw() + 
    geom_line(size=1) + scale_colour_brewer(name=expression(theta), palette="Set1") +
    theme(legend.key = element_blank()) + ggtitle("Gamma distributions with variance 4") +
    coord_cartesian(ylim = c(0, 1)) 
$\endgroup$
17
$\begingroup$

Assuming you mean "is it possible to find a probability distribution for $X$" then the answer is yes, as you have not specified any criteria that $X$ must satisfy. In fact there are an infinite number of possible distributions that would satisfy this condition. Just consider a Normal distribution, $\mathcal{N}(x ; \mu, \sigma^2)$. You can set $\sigma^2 = r$ and $\mu$ can take any value you like - you will then have $Var[X] = r$ as required.

In fact, the Normal distribution is rather special in this regard as it is the maximum entropy probability distribution for a given mean and variance.

$\endgroup$
  • $\begingroup$ you are right, I corrected it. would you please explain more? $\endgroup$ – Amir Ef Jul 2 '15 at 10:31
  • $\begingroup$ @AmirEf What is unclear? $\endgroup$ – Daniel Jul 2 '15 at 11:41
  • 6
    $\begingroup$ It's not at all clear what else Daniel should explain; the answer here seems to deal with everything in your posted question. $\endgroup$ – Glen_b -Reinstate Monica Jul 2 '15 at 13:04
15
$\begingroup$

This question can be interpreted in a way that makes it interesting and not entirely trivial. Given something $X$ that looks like a random variable, to what extent is it possible to assign probabilities to its values (or shift existing probabilities around) in such a way that its variance equals some prespecified number $r$? The answer is that all possible values $r\ge 0$ are allowable, up to a limit determined by the range of $X$.

The potential interest in such an analysis lies in the idea of changing a probability measure, while keeping a random variable fixed, in order to achieve a particular end. Although this application is simple, it displays some of the ideas underlying the Girsanov theorem, a result fundamental in mathematical finance.


Let's restate this question in a rigorous, unambiguous fashion. Suppose

$$X:(\Omega, \mathfrak{S}) \to \mathbb{R}$$

is a measurable function defined on a measure space $\Omega$ with sigma-algebra $\mathfrak{S}$. For a given real number $r \gt 0$, when is it possible to find a probability measure $\mathbb{P}$ on this space for which $\text{Var}(X) = r$?

I believe the answer is that this is possible when $\sup(X) - \inf(X) \gt 2\sqrt{r}$. (Equality can hold if the supremum and infimum are both attained: that is, they actually are the maximum and minimum of $X$.) When either $\sup(X)=\infty$ or $\inf(X)=-\infty$, this condition imposes no limit on $r$, and then all non-negative values of the variance are possible.

The proof is by construction. Let's begin with a simple version of it, to take care of the details and pin down the basic idea, and then move on to the actual construction.

  1. Let $x$ be in the image of $X$: this means there is an $\omega_x\in\Omega$ for which $X(\omega_x) = x$. Define the set function $\mathbb{P}:\mathfrak{S}\to [0,1]$ to be the indicator of $\omega_x$: that is, $\mathbb{P}(A) = 0$ if $\omega_x\notin A$ and $\mathbb{P}(A) = 1$ when $\omega_x\in A$.

    Since $\mathbb{P}(\Omega)=1$, obviously $\mathbb P$ satisfies the first two axioms of probability. It is necessary to show it satisfies the third; namely, that it is sigma-additive. But this is almost as obvious: whenever $\{E_i, i=1, 2, \ldots\}$ is a finite or countably infinite set of mutually exclusive events, then either none of them contain $\omega_x$--in which case $\mathbb{P}(E_i)=0$ for all $i$--or exactly one of them contains $\omega_x$, in which case $\mathbb{P}(E_j)=1$ for some particular $j$ and otherwise $\mathbb{P}(E_i)=0$ for all $i\ne j$. In either case

    $$\mathbb{P}\left(\cup_i E_i\right) = \sum_i \mathbb{P}(E_i)$$

    because both sides are either both $0$ or both $1$.

    Since $\mathbb{P}$ concentrates all the probability on $\omega_x$, the distribution of $X$ is concentrated on $x$ and $X$ must have zero variance.

  2. Let $x_1 \le x_2$ be two values in the range of $X$; that is, $X(\omega_1) = x_1$ and $X(\omega_2) = x_2$. In a manner similar to the previous step, define a measure $\mathbb{P}$ to be a weighted average of the indicators of $\omega_1$ and $\omega_2$. Use non-negative weights $1-p$ and $p$ for $p$ to be determined. Just as before, we find that $\mathbb{P}$--being a convex combination of the indicator measures discussed in (1)--is a probability measure. The distribution of $X$ with respect to this measure is a Bernoulli$(p)$ distribution that has been scaled by $x_2-x_1$ and shifted by $-x_1$. Because the variance of a Bernoulli$(p)$ distribution is $p(1-p)$, the variance of $X$ must be $(x_2-x_1)^2p(1-p)$.

An immediate consequence of (2) is that any $r$ for which there exist $x_1 \le x_2$ in the range of $X$ and $0 \le p \lt 1$ for which

$$r = (x_2-x_1)^2p(1-p)$$

can be the variance of $X$. Since $0 \le p(1-p) \le 1/4$, this implies

$$2\sqrt{r} = \sqrt{4 r} \le \sqrt{\frac{r}{p(1-p)}} = \sqrt{(x_2-x_1)^2} = x_2-x_1 \le \sup(X)-\inf(X),$$

with equality holding if and only if $X$ has a maximum and minimum.

Conversely, if $r$ exceeds this bound of $(\sup(X)-\inf(X))^2/4$, then no solution is possible, since we already know that the variance of any bounded random variable cannot exceed one-quarter the square of its range.

$\endgroup$
  • 3
    $\begingroup$ Dude, I think you're at a whole different level than the OP. $\endgroup$ – Mark L. Stone Jul 2 '15 at 14:46
  • 4
    $\begingroup$ @Mark Probably. (I think you detected a whiff of very dry humor here.) But anyone applying the mathematical-statistics tag to their post ought to expect this kind of stuff :-). $\endgroup$ – whuber Jul 2 '15 at 14:52
  • 2
    $\begingroup$ It kind of reminds me of when I took a 4 student class from the late Prof Samuel Karlin (of Karlin and Taylor fame among others things) on "Total Positivity". The topic of game theory somehow came up. He said, oh, game theory. You have two non-negative sigma-finite measures ...., Now imagine him introducing game theory this way to students in a freshman economics class at a liberal arts college. That's what your post made me think of. $\endgroup$ – Mark L. Stone Jul 2 '15 at 15:01
  • $\begingroup$ @Mark Understood. One would not do that and succeed. As you point out, I am writing here for (a subset of) general readers rather than a specific one. On the other hand, the abstract subject is not difficult (at this elementary level) and has proven to be accessible to motivated underclassmen at liberal arts colleges. See the comments at stats.stackexchange.com/a/94876 for instance. $\endgroup$ – whuber Jul 2 '15 at 15:13
  • 4
    $\begingroup$ @MarkL.Stone Answers are for more than just the immediate asker (SE is intended to be a repository of good questions and good answers valuable to later people with similar questions), and we have answers for the more elementary view of the question here already. Some other readers may get something out of the less elementary take on things, so a variety of styles and levels of answer makes the question useful to more people. $\endgroup$ – Glen_b -Reinstate Monica Jul 3 '15 at 0:19
10
$\begingroup$

Yes, it's possible to find such distribution. In fact you can take any distribution with a finite variance, and scale to match your condition, because $$Var[cX]=c^2Var[X]$$

For instance, a uniform distribution on interval $[0,1]$ has variance: $$\sigma^2=\frac{1}{12}$$ Hence, a uniform distribution in the interval $\left[0,\frac{1}{\sqrt{12r}}\right]$ will have variance $r$.

In fact, this is a common way to add parameters to some distributions, such as Student t. It has only one parameter, $\nu$ - degrees of freedom. When $\nu\to\infty$ the distribution converges to a standard normal. It is bell shaped, and looks a lot like normal, but has fatter tails. That's why it's often used as an alternative to a normal distribution when the tails are fat. The only problem is that Gaussian distribution has two parameters. So, comes the scaled version of Student t, which is sometimes called "t location scale" distribution. This a very simple transformation: $\xi=\frac{t-\mu}{s}$, where $\mu,s$ are location and scale. Now, you can set the scale so that the new variable $\xi$ will have any required variance, and will have a shape of Student t distribution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.