5
$\begingroup$

Suppose $X$ a random variable and $Y=f(X)$ a function of this variable

I know I can write $\mathbb V(Y)$ as $\mathbb E(f(X)^2)-\mathbb E(f(X))^2$

I would like to know if it is possible to write $\mathbb V(Y)$ as a function of $\mathbb V(X)$

e.g : $\mathbb V(Y)=g(\mathbb V(X))$ given the function $f$

In other words, I need to find an expression for the variance of the transformed variable $Y$ in which the variance of the initial variable $X$ appears explicitely, preferably as its parameter.

Details about the problem :

  1. The distribution of $X$ is unknown, and I don't think we need it because we compute variances of the whole population, not a sample of it.
  2. As it happens, the function $f$ is continuous, strictly decreasing, and positive (somewhat like $f(x)=1/x$ with $x>0$).
$\endgroup$

3 Answers 3

7
$\begingroup$

Let $X\sim \mathcal N(0,\sigma^2)$ denote a normal random variable and let $f$ be the function $$f(x) = \begin{cases}+1, & x > 0,\\-1, &x \leq 0.\end{cases}$$ Then, $f(X)$ is a random variable taking on values $\pm 1$ with equal probability and so $f(X)$ has variance $1$. On the other hand, if $X\sim \mathcal N(1,\sigma^2)$, then $f(X)$ takes on values $\pm 1$ with probabilities $\Phi\left(-\frac{1}{\sigma}\right)$ and $1-\Phi\left(-\frac{1}{\sigma}\right)$ and its variance is not $1$ even though the variance of $X$ is unchanged. Thus, it is not necessarily possible that $\mathbb V(f(X))$ can be expressed as a fixed function $g(\cdot)$ of $\mathbb V(X)$, that is, $\mathbb V(f(X))\neq g(\mathbb V(X))$.

Are there any functions $f(\cdot)$ for which $\mathbb V(f(X))$ equals $g(\mathbb V(X))$ for some fixed function $g(\cdot)$? Sure, there are. If $f(x) = ax+b$, then $$\mathbb V(f(X)) = \mathbb V(aX+b) = a^2\mathbb V(X) = g(\mathbb V(X))$$ where $g(x) = a^2x$.

What if $X$ is a positive random variable and $f(x) = x^{-1}$ for $x >0$? Well, one case where the desired relationship might hold is when if $X$ is a Gamma random variable with order parameter $3$ or more. See this answer of mine for some details of how this might be made to work.

$\endgroup$
3
$\begingroup$

The exact formula for the variance of $Y$ requires use of the function $f$ and the full distribution of $X$ (not just its variance). Nevertheless, while there is no exact formula of the kind you want, you can get approximate formulae using Taylor approximation (also called the "delta method").


To facilitate analysis using the delta method, suppose we let $\mu$, $\sigma^2$, $\gamma$ and $\kappa$ denote the mean, variance, skewness and kurtosis of $X$ respectively, and suppose that $f$ is differentiable up to the required orders for our formulae. The first-order Taylor approximation to the variance is:

$$\begin{aligned} \mathbb{V}[f(X)] &\approx f'(\mu)^2 \cdot \sigma^2. \\[6pt] \end{aligned}$$

The second-order Taylor approximation (shown in this related question and answer) is:

$$\begin{aligned} \mathbb{V}[f(X)] &\approx f'(\mu) (f'(\mu) - \mu f''(\mu)) \sigma^2 \\[6pt] &\quad - \Big[ f''(\mu)^2 \mu + (f'(\mu) - \mu f''(\mu)) f''(\mu) \Big] \gamma \sigma^3 \\[6pt] &\quad + \frac{f''(\mu)^2}{4} (\kappa-1) \sigma^4. \\[6pt] \end{aligned}$$

So, you can get an approximating formula that uses only the mean and variance of $X$ by using the first-order Taylor approximation to the variance. This is not an exact result, and it is not a particularly good approximation. If you are willing to also use the skewness and kurtosis of $X$ then you can use the second-order Taylor approximation to the variance. This is also not an exact result, but it is a reasonable approximation in a wide class of cases.

$\endgroup$
2
$\begingroup$

Let's see how far we can towards characterizing functions $f$ where such a formula will work. We know it works for linear functions, but are there any others? How about when the random variables $X$ have restricted values?


The setting of the question is one in which $f$ is given but the distribution of the random variable $X$ is unknown and arbitrary--although possibly with restrictions on the values it can assume. Thus, the appropriate sense of such a formula "working" would be

For which functions $f$ is there an associated function $V_{f}:\mathbb{R}^+\to \mathbb{R}^+$ satisfying $$V_{f}(\operatorname{Var}(X))=\operatorname{Var}(f(X))\tag{*}$$ for all random variables $X:\Omega\to \mathcal{X}\subset \mathbb{R}$?

($\Omega$ is some abstract probability space whose details don't matter.)

Let's exploit the basic properties of variance to explore these possibilities. Begin by dismissing the trivial cases where $\mathcal X$ is empty or has just one element, or when $f$ is a constant function, thereby enabling us to assume $\mathcal X$ contains two numbers $x_1$ and $x_2$ for which $f(x_2)\ne f(x_1).$ The random variables $X$ whose values are confined to this subset $\{x_1,x_2\}$ are all (at most) binary. When $\Pr(X=x_2)=p,$ direct calculation establishes $$\operatorname{Var}(X) = p(1-p)(x_2-x_1)^2.$$ The same calculation yields $$\operatorname{Var}(f(X)) = p(1-p)(f(x_2)-f(x_1))^2.$$

Keeping $(x_1,x_2)$ fixed for a moment, write $a=(x_2-x_1)^2 \gt 0$ and $b=(f(x_2)-f(x_1))^2 \gt 0.$ As $p$ varies through the interval $[0,1],$ $p(1-p)a = \lambda$ varies through the interval $[0,a/4].$ In terms of $(*),$ the preceding results tell us

$$V_f(\lambda) = V_f(\operatorname{Var}(X)) = \operatorname{Var}(f(X)) = \frac{b}{a}(\lambda).$$

This exhibits $V_f$ as a non-constant linear function defined on the interval $[0,a/4].$ Consequently, because the $x_i$ are arbitrary, $V_f$ must be a non-constant linear function defined on all values $\lambda$ from $0$ through the supremum of $(x_2-x_1)^2/4$ (which might be infinite).

This solves the problem when $\mathcal X$ has at most two elements. Suppose it has a third element $x\in\mathcal X$ distinct from $x_1$ and $x_2.$ Let $\mu$ (obviously non-negative) be the slope of $V_f$ as previously established. Thus

$$\mu = \frac{(f(x_2)-f(x))^2}{(x_2-x)^2} = \frac{(f(x_1)-f(x))^2}{(x_1-x)^2}.$$

Clearing the denominators and taking square roots produces

$$f(x_i) - f(x) = \pm \mu(x_i-x),\ i=1,2.\tag{**}$$

But it's also the case that $f(x_2) - f(x_1)=\pm \mu (x_2-x_1).$ You can (readily) check that this contradicts $(**)$ unless all the signs of all the square roots are the same. Consequently,

$f$ must be an affine function of the form $f(x) = \nu + \mu x$ for some constant numbers $\nu$ and $\mu.$

This conclusion subsumes the earlier cases where the cardinality of $\mathcal X$ is $0,$ $1,$ or $2.$

Of course--to close this logical loop--when $f$ has this form, $V_f(\lambda) = \mu^2\lambda$ is the rule for computing the variance of $f(X)$ in terms of the variance of $X$ for any random variable $X.$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.