12
$\begingroup$

How to construct an example of a probability distribution for which $\mathbb{E}\left(\frac{1}{X}\right)=\frac{1}{\mathbb{E}(X)}$ holds, assuming $\mathbb{P}(X\ne0)=1$?

The inequality which follows from Jensen's inequality for a positive-valued RV $X$ is like $\mathbb{E}\left(\frac{1}{X}\right)\ge\frac{1}{\mathbb{E}(X)}$ (the reverse inequality if $X<0$). This is because the mapping $x\mapsto\frac{1}{x}$ is convex for $x>0$ and concave for $x<0$. Following the equality condition in Jensen's inequality, I guess the distribution has to be degenerate for the required equality to hold. A trivial case where the equality holds is of course if $X=1$ a.e. Here is an example I found in a problem book: Consider a discrete random variable $X$ such that $\mathbb{P}(X=-1)=\frac{1}{9}, \mathbb{P}(X=\frac{1}{2})=\mathbb{P}(X=2)=\frac{4}{9}$. It is then easily verified that $\mathbb{E}\left(\frac{1}{X}\right)=\frac{1}{\mathbb{E}(X)}=1$.

This example shows that $X$ need not be positive (or negative) a.e. for the equality in the title to hold. The distribution here is not degenerate either.

How do I construct an example, possibly like the one I found in the book? Is there any motivation?

$\endgroup$
  • $\begingroup$ Your example does hold for any random variable that is a non-zero constant. Also your second example is not degenerate. $\endgroup$ – Michael R. Chernick Sep 30 '17 at 18:37
  • 3
    $\begingroup$ The inequality does not follow from Jensen's Inequality without further supposing that $X$ is almost surely positive. $\endgroup$ – whuber Sep 30 '17 at 19:03
  • $\begingroup$ @MichaelChernick I did not imply to say that the example had a degenerate distribution. $\endgroup$ – StubbornAtom Sep 30 '17 at 19:10
  • $\begingroup$ I was referring to your statement "Following the equality condition in Jensen's inequality, I guess the distribution has to be degenerate for the required equality to hold." Yet you showed a nondegenerate example. $\endgroup$ – Michael R. Chernick Sep 30 '17 at 19:20
  • 1
    $\begingroup$ @whuber I simply want to know how to find an example where the equality in the title is true. $\endgroup$ – StubbornAtom Sep 30 '17 at 19:55
18
$\begingroup$

Let's construct all possible examples of random variables $X$ for which $E[X]E[1/X]=1$. Then, among them, we may follow some heuristics to obtain the simplest possible example. These heuristics consist of giving the simplest possible values to all expressions that drop out of a preliminary analysis. This turns out to be the textbook example.

Preliminary analysis

This requires only a little bit of analysis based on definitions. The solution is of only secondary interest: the main objective is to develop insights to help us understand the results intuitively.

First observe that Jensen's Inequality (or the Cauchy-Schwarz Inequality) implies that for a positive random variable $X$, $E[X]E[1/X] \ge 1$, with equality holding if and only if $X$ is "degenerate": that is, $X$ is almost surely constant. When $X$ is a negative random variable, $-X$ is positive and the preceding result holds with the inequality sign reversed. Consequently, any example where $E[1/X]=1/E[X]$ must have positive probability of being negative and positive probability of being positive.

The insight here is that any $X$ with $E[X]E[1/X]=1$ must somehow be "balancing" the inequality from its positive part against the inequality in the other direction from its negative part. This will become clearer as we go along.

Consider any nonzero random variable $X$. An initial step in formulating a definition of expectation (at least when this is done in full generality using measure theory) is to decompose $X$ into its positive and negative parts, both of which are positive random variables:

$$\eqalign{ Y &= \operatorname{Positive part}(X) = \max(0, X);\\ Z &= \operatorname{Negative part}(X) = -\min(0, X). }$$

Let's think of $X$ as a mixture of $Y$ with weight $p$ and $-Z$ with weight $1-p$ where $$p=\Pr(X\gt 0),\ 1-p = \Pr(X \lt 0).$$ Obviously $$0 \lt p \lt 1.$$ This will enable us to write expectations of $X$ and $1/X$ in terms of the expectations of the positive variables $Y$ and $Z$.

To simplify the forthcoming algebra a little, note that uniformly rescaling $X$ by a number $\sigma$ does not change $E[X]E[1/X]$--but it does multiply $E[Y]$ and $E[Z]$ each by $\sigma$. For positive $\sigma$, this simply amounts to selecting the units of measurement of $X$. A negative $\sigma$ switches the roles of $Y$ and $Z$. Choosing the sign of $\sigma$ appropriately we may therefore suppose $$E[Z]=1\text{ and }E[Y] \ge E[Z].\tag{1}$$

Notation

That's it for preliminary simplifications. To create a nice notation, let us therefore write

$$\mu = E[Y];\ \nu = E[1/Y];\ \lambda=E[1/Z]$$

for the three expectations we cannot control. All three quantities are positive. Jensen's Inequality asserts

$$\mu\nu \ge 1\text{ and }\lambda \ge 1.\tag{2}$$

The Law of Total Probability expresses the expectations of $X$ and $1/X$ in terms of the quantities we have named:

$$E[X] = E[X\mid X\gt 0]\Pr(X \gt 0) + E[X\mid X \lt 0]\Pr(X \lt 0) = \mu p - (1-p) = (\mu + 1)p - 1$$

and, since $1/X$ has the same sign as $X$,

$$E\left[\frac{1}{X}\right] = E\left[\frac{1}{X}\mid X\gt 0\right]\Pr(X \gt 0) + E\left[\frac{1}{X}\mid X \lt 0\right]\Pr(X \lt 0) = \nu p - \lambda(1-p) = (\nu + \lambda)p - \lambda.$$

Equating the product of these two expressions with $1$ provides an essential relationship among the variables:

$$1 = E[X]E\left[\frac{1}{X}\right] = ((\mu +1)p - 1)((\nu + \lambda)p - \lambda).\tag{*}$$

Reformulation of the Problem

Suppose the parts of $X$--$Y$ and $Z$--are any positive random variables (degenerate or not). That determines $\mu, \nu,$ and $\lambda$. When can we find $p$, with $0 \lt p \lt 1$, for which $(*)$ holds?

This clearly articulates the "balancing" insight previously stated only vaguely: we are going to hold $Y$ and $Z$ fixed and hope to find a value of $p$ that appropriately balances their relative contributions to $X$. Although it's not immediately evident that such a $p$ need exist, what is clear is that it depends only on the moments $E[Y]$, $E[1/Y]$, $E[Z]$, and $E[1/Z]$. The problem thereby is reduced to relatively simple algebra--all the analysis of random variables has been completed.

Solution

This algebraic problem isn't too hard to solve, because $(*)$ is at worst a quadratic equation for $p$ and the governing inequalities $(1)$ and $(2)$ are relatively simple. Indeed, $(*)$ tells us the product of its roots $p_1$ and $p_2$ is

$$p_1p_2 = (\lambda - 1)\frac{1}{(\mu+1)(\nu+\lambda)} \ge 0$$

and the sum is

$$p_1 + p_2 = (2\lambda + \lambda \mu + \nu)\frac{1}{(\mu+1)(\nu+\lambda)} \gt 0.$$

Therefore both roots must be positive. Furthermore, their average is less than $1$, because

$$ 1 - \frac{(p_1+p_2)}{2} = \frac{\lambda \mu + \nu + 2 \mu \nu}{2(\mu+1)(\nu+\lambda)} \gt 0.$$

(By doing a bit of algebra, it's not hard to show the larger of the two roots does not exceed $1$, either.)

A Theorem

Here is what we have found:

Given any two positive random variables $Y$ and $Z$ (at least one of which is nondegenerate) for which $E[Y]$, $E[1/Y]$, $E[Z]$, and $E[1/Z]$ exist and are finite. Then there exist either one or two values $p$, with $0 \lt p \lt 1$, that determine a mixture variable $X$ with weight $p$ for $Y$ and weight $1-p$ for $-Z$ and for which $E[X]E[1/X]=1$. Every such instance of a random variable $X$ with $E[X]E[1/X]=1$ is of this form.

That gives us a rich set of examples indeed!


Constructing the Simplest Possible Example

Having characterized all examples, let's proceed to construct one that is as simple as possible.

  • For the negative part $Z$, let's choose a degenerate variable--the very simplest kind of random variable. It will be scaled to make its value $1$, whence $\lambda=1$. The solution of $(*)$ includes $p_1=0$, reducing it to an easily solved linear equation: the only positive root is

    $$p = \frac{1}{1+\mu} + \frac{1}{1+\nu}.\tag{3}$$

  • For the positive part $Y$, we obtain nothing useful if $Y$ is degenerate, so let's give it some probability at just two distinct positive values $a \lt b$, say $\Pr(X=b)=q$. In this case the definition of expectation gives

    $$\mu = E[Y] = (1-q)a + qb;\ \nu = E[1/Y] = (1-q)/a + q/b.$$

  • To make this even simpler, let's make $Y$ and $1/Y$ identical: this forces $q=1-q=1/2$ and $a=1/b$. Now

    $$\mu = \nu = \frac{b + 1/b}{2}.$$

    The solution $(3)$ simplifies to

    $$p = \frac{2}{1+\mu} = \frac{4}{2 + b + 1/b}.$$

  • How can we make this involve simple numbers? Since $a\lt b$ and $ab=1$, necessarily $b\gt 1$. Let's choose the simplest number greater than $1$ for $b$; namely, $b=2$. The foregoing formula yields $p = 4/(2+2+1/2) = 8/9$ and our candidate for the simplest possible example therefore is

    $$\eqalign{ \Pr(X=2) = \Pr(X=b) = \Pr(Y=b)p = qp = \frac{1}{2}\frac{8}{9} = \frac{4}{9};\\ \Pr(X=1/2) = \Pr(X=a) = \Pr(Y=a)p = qp = \cdots = \frac{4}{9};\\ \Pr(X=-1) = \Pr(Z=1)(1-p) = 1-p = \frac{1}{9}. }$$

This is the very example offered in the textbook.

$\endgroup$
  • 2
    $\begingroup$ Nice answer. Despite my initial scepticism it's easy to find an example with distinct solutions $p \in (0,1)$. $\endgroup$ – P.Windridge Oct 2 '17 at 9:07
8
$\begingroup$

As you've mentioned, if $X$ is positive then $E(1/X)=1/E(X)$ occurs only when $X$ is almost surely constant. Otherwise you need $X$ to take both negative and positive values.

To construct such an example, first go as simple as possible. Assume $X$ takes two values, $a$ and $b$, with probabilities $p$ and $1-p$ respectively. Then $$E(X)=ap+b(1-p)$$ and $$E(1/X)=\frac1ap+\frac1b(1-p).$$ To have $1/E(X)=E(1/X)$ we require $$ap+b(1-p)=\frac1{\frac1ap+\frac1b(1-p)}$$ which rearranges to the requirement $$(a-b)^2p(1-p)=0.$$ This means the only possible solution must have either $a=b$, or $p=0$, or $p=1$. In all cases we return to the degenerate case: $X$ is constant.

Next try: a distribution with three possible values. Here there are many more choices. The example you cited tries an $X$ such that $1/X$ has the same distribution. If we know $X$ takes three values, it must be that one of the values is either $1$ or $-1$, and the other two must be $a$ and $1/a$ for some choice of $a$. For definiteness let's try $P(X=a)=P(X=1/a)=p$, and $P(X=-1)=1-2p$. Then $$ E(1/X)=E(X)=(a+\frac1a)p-(1-2p)=(2+a+\frac1a)p-1.\tag1 $$ To meet the requirement $1/E(X)=E(1/X)$ we demand $E(X)=1$ or $E(X)=-1$. Expression (1) is never $-1$ unless $p=0$, which returns us to the degenerate case again. So aim for $E(X)=1$, which gives $$(2+a+\frac1a)p=2\quad\Leftrightarrow\quad p=\frac2{2+a+\frac1a}=\frac{2a}{(a+1)^2}.\tag2$$ Expression (2) gives an entire family of solutions that meet the requirement. The only constraint is that $a$ must be positive. The example you cited takes $a=2$. Only the case $a=1$ is degenerate.

$\endgroup$
  • 1
    $\begingroup$ I assume your first line is intended to be "if $X$ is positive then $\mathbb{E}[1/X]=1/\mathbb{E}[X]$ occurs only when $X$ is almost surely constant", like in the question. This drops out from (the proof of) Jensen's inequality, where we also use the fact that $h(x) = 1/x$ is not linear. $\endgroup$ – P.Windridge Oct 1 '17 at 10:39
  • $\begingroup$ @P.Windridge You are right! Fixed. $\endgroup$ – grand_chat Oct 1 '17 at 17:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.