24
$\begingroup$

The title is the question. I am told that ratios and inverses of random variables often are problematic. What is meant is that expectation often do not exist. Is there a simple, general explication of that?

$\endgroup$
22
$\begingroup$

I would like to offer a very simple, intuitive explanation. It amounts to looking at a picture: the rest of this post explains the picture and draws conclusions from it.

Here is what it comes down to: when there is a "probability mass" concentrated near $X=0$, there will be too much probability near $1/X\approx \pm \infty$, causing its expectation to be undefined.


Instead of being fully general, let's focus on random variables $X$ that have continuous densities $f_X$ in a neighborhood of $0$. Suppose $f_X(0)\ne 0$. Visually, these conditions mean the graph of $f$ lies above the axis around $0$:

Figure showing the graph of a density and the area below it.

The continuity of $f_X$ around $0$ implies that for any positive height $p$ less than $f_X(0)$ and sufficiently small $\epsilon$, we may carve out a rectangle beneath this graph which is centered around $x=0$, has width $2\epsilon$, and height $p$, as shown. This corresponds to expressing the original distribution as a mixture of a uniform distribution (with weight $p\times 2\epsilon=2p\epsilon$) and whatever remains.

Figure showing the graph as a mixture.

In other words, we may think of $X$ as arising in the following way:

  1. With probability $2p\epsilon$, draw a value from a Uniform$(-\epsilon,\epsilon)$ distribution.

  2. Otherwise, draw a value from the distribution whose density is proportional to $f_X - p I_{(-\epsilon,\epsilon)}$. (This is the function drawn in yellow at the right.)

($I$ is the indicator function.)

Step $(1)$ shows that for any $0 \lt u \lt \epsilon$, the chance that $X$ is between $0$ and $u$ exceeds $p u / 2$. Equivalently, this is the chance that $1/X$ exceeds $1/u$. To put it another way: writing $S$ for the survivor function of $1/X$

$$S(x) = \Pr(1/X \gt x),$$

the picture shows $S(x) \gt p / (2x)$ for all $x \gt 1/\epsilon$.

We're done now, because this fact about $S$ implies the expectation is undefined. Compare the integrals involved in computing the expectation of the positive part of $1/X$, $(1/X)_{+} = \max(0, 1/X)$:

$$E[(1/X)_{+}] = \int_0^\infty S(x)dx \gt \int_{1/\epsilon}^x S(x)dx \gt \int_{1/\epsilon}^x \frac{p}{2x}dx = \frac{p}{2} \log(x\epsilon).$$

(This is a purely geometric argument: every integral represents an identifiable two-dimensional region and all the inequalities arise from strict inclusions within those regions. Indeed, we don't even need to know the final integral is a logarithm: there are simple geometric arguments showing this integral diverges.)

Since the right side diverges as $x\to\infty$, $E[(1/X)_{+}]$ diverges, too. The situation with the negative part of $1/X$ is the same (because the rectangle is centered around $0$), and the same argument shows the expectation of the negative part of $1/X$ diverges. Consequently the expectation of $1/X$ itself is undefined.

Incidentally, the same argument shows that when $X$ has probability concentrated on one side of $0$, such as any Exponential or Gamma distribution (with shape parameter less than $1$), then still the positive expectation diverges, but the negative expectation is zero. In this case the expectation is defined, but is infinite.

$\endgroup$
  • 2
    $\begingroup$ Am I right in suspecting that the assumption $f_X(0)\neq 0$ is crucial for the result? I mean, we have cases where $1/X$ has moments at least for some range of involved parameters, and it appears that it is in cases where $f_X(0) = 0$, like Gamma/Inverse-Gamma $\endgroup$ – Alecos Papadopoulos Aug 26 '17 at 20:00
  • 3
    $\begingroup$ @Alecos No, that assumption is not crucial. That and the continuity of $f$ at $0$ make the argument simple, but neither is essential. Consider an $X$ with density $f_X$ proportional to $-1/\log(x)$ for $0 \lt x \lt 1/e$ and $f_X(0)=0$. This is continuous at $0$ but $1/X$ has no expectation. $\endgroup$ – whuber Aug 27 '17 at 20:35
14
$\begingroup$

Ratios and inverses are mostly meaningful with nonnegative random variables, so I will assume $X \ge 0$ almost surely. Then, if $X$ is a discrete variable which take on the value zero with positive probability, we will be dividing with zero with a positive probability, which explains why the expectation of $1/X$ will not exist.

Now look at the continuous distribution case, with $X \ge 0$ a random variable with density function $f(x)$. We will assume that $f(0)>0$ and that $f$ is continuous (at least at zero). Then there is an $\epsilon > 0$ such that $f(x) > \epsilon $ for $0 \le x < \epsilon$. The expected value of $1/X$ is given by $$ \DeclareMathOperator{\E}{\mathbb{E}} \E \frac1{X} = \int_0^\infty \frac1{x} f(x)\; dx $$ Now let us change variable of integration to $u=1/x$, we have $du = -\frac1{x^2} \; dx$, obtaining $$ \E \frac1{X} = -\int_{\infty}^0 u f(\frac1{u}) (\frac1{u})^2 \; du = \\ \int_0^\infty \frac1{u} f(\frac1{u}) \; du $$ Now, by assumption $f(u) > \epsilon$ on $[0,\epsilon)$ so $f(\frac1{u}) > 1/\epsilon$ on $(1/\epsilon, \infty)$, using this we have $$ \E \frac1{X} > \epsilon \int_{1/\epsilon}^\infty \frac1{u}\; du =\infty $$ showing that the expectation does not exist. An example fulfilling this assumption is the exponential distribution with rate 1.

We have given an answer for inverses, what about ratios? Let $Z=Y/X$ be the ratio of two nonnegative random variables. If they are independent, we can write $$ \E Z = \E\frac{Y}{X}=\E Y \cdot \E\frac1{x} $$ so this pretty much reduces to the first case and there is not much new to say. What if they are dependent, with joint density factoring as $$ f(x,y) = f(x \mid y) g(y) $$ Then we get (using same substitution as above) $$ \E \frac{Y}{X} = \int_0^\infty y \int_0^\infty \frac1{x} f(x\mid y) \; dx \;g(y)\; dy = \\ \int_0^\infty y \int_0^\infty \frac1{u} f(\frac1{u}\mid y) \; du \; g(y) \; dy $$ and we can reason as above on the inner integral. The result will be that if the conditional density (given $y$) is positive and continuous at zero, for a set of $y$'s with positive marginal probability, the expectation will be infinite. I guess it will not be easy to find examples where the marginal expectation of $1/X$ is infinite, but the expectation of the ratio $Y/X$ is finite, unless there is a perfect correlation. It would be nice to see some such examples!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.