Mean and variance of the reciprocal of a random variable

Question

My random variable $X$ is normally distributed with mean $b$ and variance $p$. I defined a new random variable, $Y$, such that $Y=\frac{1}{X}$. Does anyone know how to find the mean of $Y$ and $Y^2$?

Answer 1

Actually, I wrote a paper in 1991 addressing this issue, available on Research Gate. In short, the mean and variance do not exist.

More precisely, if one defines a generalised inverse normal distribution by its density ($\alpha>1,\sigma>0$) $$f(z|\mu,\sigma,\alpha) = \dfrac{K(\alpha,\mu,\sigma)}{|z|^\alpha}\exp\left\{-(z^{-1}-\mu)^2/2\sigma^2\right\}$$ then $$K(\alpha,\mu,\sigma)^{-1}=\sigma^{\alpha-1}e^{-\mu^2/2\sigma^2}2^{(\alpha-1)/2}\Gamma((\alpha-1)/2){}_1F_1\left(\frac{1}{2}(\alpha-1);\frac{1}{2};\frac{\mu^2}{2\sigma^2}\right)$$ where ${}_1F_1$ is a confluent hypergeometric function. From this derivation of the normalising constant, one deduces that the mean only exists for $\alpha>2$ (while the inverse normal distribution corresponds to $\alpha=2$) and the variance only exists for $\alpha>3$. The mean is given by $$\mathbb{E}[X]=\frac{\mu}{\sigma^2}\,\frac{{}_1F_1\left(\frac{1}{2}(\alpha-3);\frac{3}{2};\frac{\mu^2}{2\sigma^2}\right)}{{}_1F_1\left(\frac{1}{2}(\alpha-1);\frac{1}{2};\frac{\mu^2}{2\sigma^2}\right)}$$

A much simpler argument as to why the mean does not exist is that, if $f$ is the normal $\text{N}(b,p)$ density, $$x^{-1} f(x)\equiv \frac{e^{-b^2/2p}}{\sqrt{2\pi p}}x^{-1}$$ at zero, which is not integrable.

Answer 2

This is equivalent to asking for $\int_{- \infty}^\infty \frac{1}{x} P(X=x) \, dx$ and $\int_{- \infty}^\infty \frac{1}{x^2} P(X=x) \, dx$.

Considering the second, let's choose some arbitrary $a > 0$: $$\eqalign { \int_{- \infty}^\infty \frac{1}{x^2} P(X=x) \, dx &= \int_{- \infty}^{-a} \frac{1}{x^2} P(X=x) \, dx\\ &+ \int_{-a}^a \frac{1}{x^2} P(X=x) \, dx + \int_a^\infty \frac{1}{x^2} P(X=x) \, dx \\ &\ge \int_{-a}^a \frac{1}{x^2} P(X=x) \, dx \\ &\ge \int_{-a}^a \frac{1}{x^2} \min\left(P(X=-a), P(X=a)\right) \, dx \\ &= \min\left(P(X=-a), P(X=a)\right) \int_{-a}^a \frac{1}{x^2} \, dx \to \infty }$$ where first we split up the integral and note that the tails are nonnegative, and then use the fact that the normal pdf is quasiconcave. So $\mathbb{E}\left[\frac{1}{X^2}\right]$ doesn't exist.

We can do exactly the same thing to show that $\mathbb{E}\left[\frac{1}{\lvert X \rvert}\right]$ doesn't exist (since $\int_{0}^a \frac{1}{X} dx$ also diverges). But $\frac{1}{X}$ is a little trickier, since one of the tail integrals is negative-valued. Still, $\mathbb{E}[\frac{1}{X} \mid \frac{1}{X} < a]$ doesn't exist, since the integral diverges. I think that, because $P(\frac{1}{X} < a) > 0$, using basically the same argument as the law of total expectation, this then implies that the overall expectation cannot exist. But I'm not 100% on that.

Answer 3

It is true that, as others have said, the means of $Y$ and $Y^{2}$ ($Y=1/X$, where $X$ is normally distributed with mean $\mu$ and standard deviation $\sigma$) don't exist as ordinary integrals. Nevertheless, the mean of $Y$ exists in the principal value sense, and can be expressed in terms of a well-understood special function, the Dawson function.

The mean of $Y^{2}$ also seems to exist in some generalized sense, and can also be expressed in terms of the Dawson function. But this generalized sense, whatever it is, is more convoluted than just the principal value.

The analytic expressions for the means of $Y$ and $Y^{2}$ can be compared to the results one gets from (pseudo-) random sampling from the normal distribution, and I will do that below. One draws the values of $X$ from the normal distribution, and computes the values $1/X$ and $1/X^{2}$. When one computes the average of all the $1/X$ values, one finds excellent agreement with the prediction of the analytic expression (as long as $\mu$ is not too close to zero). And similarly, when one computes the average of all the $1/X^{2}$ values, one finds excellent agreement with the prediction of the analytic expression (again, as long as $\mu$ is not too close to zero).

That the mean of $Y$ is expressible as a principal value integral is explained e.g. on Wikipedia, but I will summarize it here (and add a few extra bits of information). I will also argue that taking the principal value is entirely consistent with the interpretation of the integral as a probabilistic mean value.

As far as why the mean of $Y^{2}$ is also ultimately computable, I will unfortunately only be able to give a heuristic argument.

The analytic results

$$\text{P.V.}\;\mathbb{E}[Y]=\text{P.V.}\;\int_{-\infty}^{\infty}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx=\lim_{\epsilon\to 0^{+}}\left(\int_{-\infty}^{-\epsilon}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx+\int_{\epsilon}^{\infty}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx\right)$$ $$=\frac{\sqrt{2}}{\sigma}F\left(\frac{\mu}{\sqrt{2}\,\sigma}\right)\,, $$ and $$\mathbb{E}[Y^2]\overset{\overset{\scriptstyle\text{some}}{\scriptstyle\text{sense?}}}{=}\frac{1}{\sigma^{2}}\left(\mu\,\mathbb{E}[Y]-1\right)=\frac{1}{\sigma^{2}}\left(\frac{\sqrt{2}\mu}{\sigma}F\left(\frac{\mu}{\sqrt{2}\,\sigma}\right)-1\right)=-\frac{1}{\sigma^{2}}F'\left(\frac{\mu}{\sqrt{2}\,\sigma}\right)\,. $$

In the expressions above, $F(x)$ is the Dawson function $$ F(x)=e^{-x^{2}}\int_{0}^{x}e^{t^{2}}dt $$ for which efficient evaluation algorithms exist (it is for example implemented in Mathematica as DawsonF, and you can evaluate it on Wolfram Alpha, like so). Also, $F'(x)$ is its derivative, which satisfies $F'(x)=1-2xF(x)$. Among other useful properties are that $F(-x)=-F(x)$ and $F'(-x)=F'(x)$.

Series expansions

One can also use the following series expansions:

$$\text{P.V.}\;\mathbb{E}[Y]=\frac{1}{\mu}+\frac{\sigma^{2}}{\mu^{3}}+\frac{3\sigma^{4}}{\mu^{5}}+\frac{15 \sigma^{6}}{\mu^{7}}+\frac{105\sigma^{8}}{\mu^{9}}+\frac{945 \sigma^{10}}{\mu^{11}}+\cdots\hspace{2.5em}$$ $$\mathbb{E}[Y^{2}]\overset{\overset{\scriptstyle\text{some}}{\scriptstyle\text{sense?}}}{=}\frac{1}{\mu^{2}}+\frac{3\sigma^{2}}{\mu^{4}}+\frac{15\sigma^{4}}{\mu^{6}}+\frac{105 \sigma^{6}}{\mu^{8}}+\frac{945\sigma^{8}}{\mu^{10}}+\frac{10395 \sigma^{10}}{\mu^{12}}+\cdots$$

Comparison to the results from random sampling

All the results above can be tested (e.g. in Mathematica) by comparing them to the mean values computed from (pseudo-) random samples. For $\mu=0.9$ and $\sigma=0.1$, with $N=10^{8}$ sample points, I get the following results:

                                    $\mathbb{E}[Y]$                $\mathbb{E}[Y^{2}]$
analytic                 1.125 371 01      1.283 390 59
series                   1.125 370 96      1.283 389 86
random sample    1.125 377 74      1.283 410 66

As far as the integral defining $\mathbb{E}[Y]$, at this point, I would like to argue that it makes sense that we take that integral in the principal value sense. In a nutshell, the mean value computed from a random sample is precisely what we would get if we performed Monte Carlo integration of the relevant integral, provided that the integral is taken in the principal value sense. This is because the Monte Carlo sampling does not proceed first separately below the singularity, then separately above. It is rather constantly sampling both above and below the singularity.

To put it another way, recall that taking the principal value makes it so that the divergence as one approaches the singularity from the left exactly cancels the divergence as one approaches the singularity from the right. But taking random samples will accomplish the same thing, since the samples are, in an appropriate limiting sense, equally likely to come from either side of the singularity. In other words, we are about equally likely to randomly draw $X$ values that are very slightly above zero and those that are very slightly below zero; and so we are about equally likely to get values of $Y=1/X$ that are very large in magnitude and positive and those that are very large in magnitude and negative. I say 'about equally likely', but the larger the magnitudes, the closer become the two likelihoods (of drawing the negative and positive values of that magnitude). In this way, the mean value remains finite.

Failure when $\mu$ is too close to zero

Once $\mu$ starts to get too close to zero, one would of course expect something to go bad. Indeed, the results for random sampling start to fluctuate widely from sample to sample (even with samples of $10^{8}$ points). In my samples, for $\sigma=0.1$, this happens for $\mathbb{E}[Y^{2}]$ when $|\mu|$ is below about 0.5, and for $\mathbb{E}[Y]$ when $|\mu|$ is below about 0.2.

On the analytical side, one obvious thing to notice is that, for $|\mu|$ low enough, the value of $-\frac{1}{\sigma^{2}}F'\left(\frac{\mu}{\sqrt{2}\,\sigma}\right)$ (which is 'in some sense' equal to $\mathbb{E}[Y^2]$) turns negative; in particular, note that $F'(0)=1$. This is obviously non-sensical. Moreover, as $|\mu|$ approaches zero, the magnitude of $\text{P.V.}\;\mathbb{E}[Y]=\frac{\sqrt{2}}{\sigma}F\left(\frac{\mu}{\sqrt{2}\,\sigma}\right) $ reaches a maximum and then decreases, reaching zero at $\mu=0$; this, too, is non-sensical.

So the analytical results are letting us know that they can't be trusted when $|\mu|$ is too small, but they don't tell us how small a $|\mu|$, precisely, is too small. They also don't tell us what the nature of the failure is, exactly. Finally, the derivation of the analytical results doesn't offer (at least to me) any reason to predict that small values of $|\mu|$ are in any way dangerous.

Derivation of the analytic formulas

The mean value of $Y$

By changing the variables of integration, we have that $$\text{P.V.}\;\int_{-\infty}^{\infty}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx=-\frac{1}{\sqrt{2\pi}\,\sigma}\;\left(\text{P.V.}\;\int_{-\infty}^{\infty}\frac{1}{-\frac{\mu}{\sqrt{2}\sigma}-y}e^{-y^{2}}dy\right)$$ $$=-\frac{\pi}{\sqrt{2\pi}\,\sigma}\;H\left(-\frac{\mu}{\sqrt{2}\sigma}\right)\,,$$ where $H(z)$ is is the Hilbert transform of a Gaussian: $$ H(z)=\frac{1}{\pi}\;\text{P.V.}\;\int_{-\infty}^{\infty}\frac{e^{-x^{2}}}{z-x}dx\,. $$ The Hilbert transform of a Gaussian, in turn, is expressible in terms of the Dawson function (for a derivation, see e.g. here, or this paper): $$ H(z)=\frac{2}{\sqrt{\pi}}F(z)\,. $$ The result is the formula given above.

The mean value of $Y^{2}$

This is trickier, and I can only offer a heuristic derivation. Notice that, formally speaking, the integral $$\int_{-\infty}^{\infty}\frac{1}{x^{2}}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx$$ doesn't exist even in the principal value sense: the divergence around zero is positive both below and above zero, so there is no way for the two infinite areas to cancel.

Heuristically, however, we can proceed as follows. We begin with the following change of variables, which is nonproblematic:

$$\text{P.V.}\;\int_{-\infty}^{\infty}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx = \text{P.V.}\;\int_{-\infty}^{\infty}\frac{1}{u+\mu}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{u^{2}}{2\sigma^{2}}}du$$

Now we note that a formal differentiation of the two sides of this equality with respect to $\mu$ gives, on the right-hand side, $$\frac{d}{d\mu} \int_{-\infty}^{\infty}\frac{1}{u+\mu}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{u^{2}}{2\sigma^{2}}}du=-\int_{-\infty}^{\infty}\frac{1}{(u+\mu)^{2}}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{u^{2}}{2\sigma^{2}}}du$$ $$=-\int_{-\infty}^{\infty}\frac{1}{x^{2}}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx\,.$$

This last expression (if the integral existed!) would be $-\mathbb{E}[Y^{2}]$.

And on the left-hand side, a formal differentiation gives $$\frac{d}{d\mu}\int_{-\infty}^{\infty}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx=\frac{1}{\sigma^{2}}\left(1-\mu\int_{-\infty}^{\infty}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx\right).$$

The last expression becomes $\frac{1}{\sigma^{2}}\left(1-\mu\,\mathbb{E}[Y]\right)$, provided we (all of the sudden) again take the integral in the principal value sense. In this heuristic way, we get that $$\mathbb{E}[Y^{2}]=\frac{1}{\sigma^{2}}\left(\mu\,\mathbb{E}[Y]-1\right)\,.$$

Just to illustrate some of the difficulties ignored in this heuristic approach, let us consider what happens if we try to treat the principal values honestly. Then, when we differentiate the right-hand side above, we must write

$$\frac{d}{d\mu} \lim_{\epsilon\to 0^{+}} \left(\int_{-\infty}^{-\mu-\epsilon}\frac{1}{u+\mu}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{u^{2}}{2\sigma^{2}}}du+\int_{-\mu+\epsilon}^{\infty}\frac{1}{u+\mu}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{u^{2}}{2\sigma^{2}}}du\right).$$

Let's assume that we can move the differentiation inside the limit (I don't have a proof that this is OK, but probably it actually is). We get $$\lim_{\epsilon\to 0^{+}} \left[\frac{1}{\epsilon}\frac{1}{\sqrt{2\pi}\sigma}\left(e^{-\frac{(\mu+\epsilon)^{2}}{2\sigma^{2}}}+e^{-\frac{(\mu-\epsilon)^{2}}{2\sigma^{2}}}\right)\right.$$ $$\left.-\int_{-\infty}^{-\mu-\epsilon}\frac{1}{(u+\mu)^{2}}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{u^{2}}{2\sigma^{2}}}du-\int_{-\mu+\epsilon}^{\infty}\frac{1}{(u+\mu)^{2}}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{u^{2}}{2\sigma^{2}}}du\right]\,.$$ The first term, which comes from the differentiation of the limits of integration, diverges (when $\epsilon\to 0$) as $\frac{1}{\epsilon}\frac{2}{\sqrt{2\pi}\sigma}e^{-\frac{\mu^{2}}{2\sigma^{2}}}$, and presumably exactly cancels the divergences from the two integrals that follow. (Borrowing terminology from particle physics, this divergence-cancelling term might be called a 'counterterm'.) So the limit as a whole may well exist.

However, it is not clear why this rather complicated 'limit as a whole' should be identified with $-\mathbb{E}[Y^{2}]$. True, the two integrals are suggestive of it (once we change variables to $x=u+\mu$), but we have no interpretation for the counterterm.

So, our heuristic argument indeed ignored some serious difficulties. Nevertheless, as we have seen above, there is something right about all this, because the analytic result for $\mathbb{E}[Y^{2}]$ agrees extremely well with the results obtained from random samples (as long as $\mu$ is not too close to zero).

Derivation of series expansions

A simple change of variables gives

$$\text{P.V.}\;\mathbb{E}[Y]=\text{P.V.}\;\int_{-\infty}^{\infty}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx=\text{P.V.}\;\int_{-\infty}^{\infty}\frac{1}{\mu+y}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{y^{2}}{2\sigma^{2}}}dy$$

We use the series expansion

$$\frac{1}{\mu+y}=\frac{1}{\mu}-\frac{y}{\mu^2}+\frac{y^2}{\mu^3}-\frac{y^3}{\mu^4}+\cdots$$

and integrate term by term. The result is given above.

For the series expansion of $\mathbb{E}[Y^{2}]$, we start with a similar (but merely 'formal') expression $$\mathbb{E}[Y^{2}]=\int_{-\infty}^{\infty}\frac{1}{(\mu+y)^{2}}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{y^{2}}{2\sigma^{2}}}dy\,.$$ As we've noted bove, this integral doesn't exist even in the principal value sense; it requires a 'counterterm'. Nevertheless, inside the integral, we insert a series expansion of $1/(\mu+y)^{2}$, and formally integrate term by term. We obtain the result quoted above. And as we've seem, the resulting expansion works extremely well (when compared to both the analytic result and the result from random samples), as long as $\mu$ is not too close to zero.