31
$\begingroup$

The title is the question. I am told that ratios and inverses of random variables often are problematic. What is meant is that expectation often do not exist. Is there a simple, general explication of that?

$\endgroup$
33
$\begingroup$

I would like to offer a very simple, intuitive explanation. It amounts to looking at a picture: the rest of this post explains the picture and draws conclusions from it.

Here is what it comes down to: when there is a "probability mass" concentrated near $X=0$, there will be too much probability near $1/X\approx \pm \infty$, causing its expectation to be undefined.


Instead of being fully general, let's focus on random variables $X$ that have continuous densities $f_X$ in a neighborhood of $0$. Suppose $f_X(0)\ne 0$. Visually, these conditions mean the graph of $f$ lies above the axis around $0$:

Figure showing the graph of a density and the area below it.

The continuity of $f_X$ around $0$ implies that for any positive height $p$ less than $f_X(0)$ and sufficiently small $\epsilon$, we may carve out a rectangle beneath this graph which is centered around $x=0$, has width $2\epsilon$, and height $p$, as shown. This corresponds to expressing the original distribution as a mixture of a uniform distribution (with weight $p\times 2\epsilon=2p\epsilon$) and whatever remains.

Figure showing the graph as a mixture.

In other words, we may think of $X$ as arising in the following way:

  1. With probability $2p\epsilon$, draw a value from a Uniform$(-\epsilon,\epsilon)$ distribution.

  2. Otherwise, draw a value from the distribution whose density is proportional to $f_X - p I_{(-\epsilon,\epsilon)}$. (This is the function drawn in yellow at the right.)

($I$ is the indicator function.)

Step $(1)$ shows that for any $0 \lt u \lt \epsilon$, the chance that $X$ is between $0$ and $u$ exceeds $p u / 2$. Equivalently, this is the chance that $1/X$ exceeds $1/u$. To put it another way: writing $S$ for the survivor function of $1/X$

$$S(x) = \Pr(1/X \gt x),$$

the picture shows $S(x) \gt p / (2x)$ for all $x \gt 1/\epsilon$.

We're done now, because this fact about $S$ implies the expectation is undefined. Compare the integrals involved in computing the expectation of the positive part of $1/X$, $(1/X)_{+} = \max(0, 1/X)$:

$$E[(1/X)_{+}] = \int_0^\infty S(x)dx \gt \int_{1/\epsilon}^x S(x)dx \gt \int_{1/\epsilon}^x \frac{p}{2x}dx = \frac{p}{2} \log(x\epsilon).$$

(This is a purely geometric argument: every integral represents an identifiable two-dimensional region and all the inequalities arise from strict inclusions within those regions. Indeed, we don't even need to know the final integral is a logarithm: there are simple geometric arguments showing this integral diverges.)

Since the right side diverges as $x\to\infty$, $E[(1/X)_{+}]$ diverges, too. The situation with the negative part of $1/X$ is the same (because the rectangle is centered around $0$), and the same argument shows the expectation of the negative part of $1/X$ diverges. Consequently the expectation of $1/X$ itself is undefined.

Incidentally, the same argument shows that when $X$ has probability concentrated on one side of $0$, such as any Exponential or Gamma distribution (with shape parameter less than $1$), then still the positive expectation diverges, but the negative expectation is zero. In this case the expectation is defined, but is infinite.

$\endgroup$
2
  • 3
    $\begingroup$ Am I right in suspecting that the assumption $f_X(0)\neq 0$ is crucial for the result? I mean, we have cases where $1/X$ has moments at least for some range of involved parameters, and it appears that it is in cases where $f_X(0) = 0$, like Gamma/Inverse-Gamma $\endgroup$ Aug 26 '17 at 20:00
  • 5
    $\begingroup$ @Alecos No, that assumption is not crucial. That and the continuity of $f$ at $0$ make the argument simple, but neither is essential. Consider an $X$ with density $f_X$ proportional to $-1/\log(x)$ for $0 \lt x \lt 1/e$ and $f_X(0)=0$. This is continuous at $0$ but $1/X$ has no expectation. $\endgroup$
    – whuber
    Aug 27 '17 at 20:35
15
$\begingroup$

Ratios and inverses are mostly meaningful with nonnegative random variables, so I will assume $X \ge 0$ almost surely. Then, if $X$ is a discrete variable which take on the value zero with positive probability, we will be dividing with zero with a positive probability, which explains why the expectation of $1/X$ will not exist.

Now look at the continuous distribution case, with $X \ge 0$ a random variable with density function $f(x)$. We will assume that $f(0)>0$ and that $f$ is continuous (at least at zero). Then there is an $\epsilon > 0$ such that $f(x) > \epsilon $ for $0 \le x < \epsilon$. The expected value of $1/X$ is given by $$ \DeclareMathOperator{\E}{\mathbb{E}} \E \frac1{X} = \int_0^\infty \frac1{x} f(x)\; dx $$ Now let us change variable of integration to $u=1/x$, we have $du = -\frac1{x^2} \; dx$, obtaining $$ \E \frac1{X} = -\int_{\infty}^0 u f(\frac1{u}) (\frac1{u})^2 \; du = \\ \int_0^\infty \frac1{u} f(\frac1{u}) \; du $$ Now, by assumption $f(u) > \epsilon$ on $[0,\epsilon)$ so $f(\frac1{u}) > 1/\epsilon$ on $(1/\epsilon, \infty)$, using this we have $$ \E \frac1{X} > \epsilon \int_{1/\epsilon}^\infty \frac1{u}\; du =\infty $$ showing that the expectation does not exist. An example fulfilling this assumption is the exponential distribution with rate 1.

We have given an answer for inverses, what about ratios? Let $Z=Y/X$ be the ratio of two nonnegative random variables. If they are independent, we can write $$ \E Z = \E\frac{Y}{X}=\E Y \cdot \E\frac1{x} $$ so this pretty much reduces to the first case and there is not much new to say. What if they are dependent, with joint density factoring as $$ f(x,y) = f(x \mid y) g(y) $$ Then we get (using same substitution as above) $$ \E \frac{Y}{X} = \int_0^\infty y \int_0^\infty \frac1{x} f(x\mid y) \; dx \;g(y)\; dy = \\ \int_0^\infty y \int_0^\infty \frac1{u} f(\frac1{u}\mid y) \; du \; g(y) \; dy $$ and we can reason as above on the inner integral. The result will be that if the conditional density (given $y$) is positive and continuous at zero, for a set of $y$'s with positive marginal probability, the expectation will be infinite. I guess it will not be easy to find examples where the marginal expectation of $1/X$ is infinite, but the expectation of the ratio $Y/X$ is finite, unless there is a perfect correlation. It would be nice to see some such examples!

$\endgroup$
1
  • 1
    $\begingroup$ Re your last remark: such examples abound. Take $U$ to be any variable with infinite expectation and $V$ be an independent variable with finite expectation. Set $X=1/U$ and $Y=V/U,$ so that $(1/X,Y/X)=(U,V).$ Want the correlation to be small? Then let $B$ be an independent Rademacher variable and set $Y=BV/U$ instead; now, any measure of "correlation" must be zero due to the symmetry of $Y.$ $\endgroup$
    – whuber
    Mar 17 at 22:08
2
$\begingroup$

Let's offer a "dissenting" view:
Ratios and inverses of random variables can be fine in the following sense:

  • It may be the case that in many cases they do not possess moments
  • But it is also the case that in many cases they result in recognizable, "named" and exhaustively studied distributions.
  • ...and there is distribution-life beyond moments, like probabilities and quantiles

EXAMPLES for RATIOS

  • Student's t-distribution is the ratio of a Normal and a Chi distribution
  • F-distribution is the ratio of two Chi-squares
  • Ratio of two Normals is Cauchy
  • Ratio of two Exponentials is Lomax (shifted Pareto)
    etc

On the contrary, it is products of random variables that appear to not lead to recognizable distributions that often.

$\endgroup$
4
  • $\begingroup$ Many of your examples are non-examples: for instance, the F distribution has infinite mean when its second df parameter is $2$ or smaller; the Cauchy distribution (same as Student's t with 1 df) has no mean; Pareto distributions with sufficiently long tails have no mean. $\endgroup$
    – whuber
    Mar 17 at 22:03
  • $\begingroup$ @whuber I don't understand your point. The first thing I state in my answer is that ratios often don't have moments, and then I explain why they are more "friendly" than products, regardless. $\endgroup$ Mar 18 at 1:51
  • $\begingroup$ What, then, do you mean by "examples"? What are these intended to be examples of? $\endgroup$
    – whuber
    Mar 18 at 13:28
  • $\begingroup$ @whuber Examples of ratios of random variables that map to well-known and well-studied distributions, and so not so "problematic" as regards their usability, in reference to the word that the OP uses in the title of the post. $\endgroup$ Mar 18 at 13:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.