# Mean and variance of the reciprocal of a random variable

My random variable $$X$$ is normally distributed with mean $$b$$ and variance $$p$$. I defined a new random variable, $$Y$$, such that $$Y=\frac{1}{X}$$. Does anyone know how to find the mean of $$Y$$ and $$Y^2$$?

• Sep 15, 2013 at 2:08
• ctd... for example, the Taylor expansion to first order suggests (assuming I didn't make an error) that $\text{Var}(1/X)$ should be approximately $\text{Var}(X)/\mu^4$. Taking $\sqrt p = .1$, and $b=10$ (CV=0.01), we get the approximate answer of $\text{Var}(1/X)\approx 1\times 10^{-6}$, and simulating in R (set.seed(239326);x=rnorm(1000,10,.1);y=1/x;var(y)) we obtain 1.05e-6. It works pretty well under a variety of values for $p$ and $b$, as long as the CV is small. If Taylor expansion doesn't work, we're left to account for its success under the conditions I suggested above. Sep 15, 2013 at 23:22
• It may be that we're dealing with a case where the expectation of the remainder isn't small but the remainder itself is nearly always small (e.g. where the middle 99.99...% of the distribution of the remainder is small); that could yield a situation where the low-order Taylor approximation is highly useful in practice (in that, essentially every time you try it with small coefficient of variation, it actually works quite well), even though it doesn't work 'on average' because of a extremely small chance of an even more extremely large result. Sep 15, 2013 at 23:58
• Dear @Glen_b: From a mathematical point of view, the Taylor series doesn't work, regardless of the coefficient of variation. Since this is a question about a mathematical property, I think it's important to point this out. Regarding "practicalities", I think that, in certain circumstances, one could easily argue the opposite---the outlined approach could be highly dangerous precisely because it appears like it might work the vast majority of the time. But, when it doesn't, the error is catastrophic. (The relevance of such an argument would depend on the application, of course.) Sep 16, 2013 at 0:46
• @cardinal Certainly the required expectations don't exist (and the Taylor series for the expectations cannot work, as you say), which is all that need be said for the question at hand. My remaining questions about why the sample moments appear to work so well should go elsewhere (and will clarify how to modify at least one answer of mine elsewhere), but will have to wait until I can formulate a better question. I'm uncertain which comments should be deleted and which should remain (here and elsewhere); I'm happy to leave those where the explanatory responses would be relevant to others. Sep 16, 2013 at 2:11

Actually, I wrote a paper in 1991 addressing this issue, available on Research Gate. In short, the mean and variance do not exist.

More precisely, if one defines a generalised inverse normal distribution by its density ($\alpha>1,\sigma>0$) $$f(z|\mu,\sigma,\alpha) = \dfrac{K(\alpha,\mu,\sigma)}{|z|^\alpha}\exp\left\{-(z^{-1}-\mu)^2/2\sigma^2\right\}$$ then $$K(\alpha,\mu,\sigma)^{-1}=\sigma^{\alpha-1}e^{-\mu^2/2\sigma^2}2^{(\alpha-1)/2}\Gamma((\alpha-1)/2){}_1F_1\left(\frac{1}{2}(\alpha-1);\frac{1}{2};\frac{\mu^2}{2\sigma^2}\right)$$ where ${}_1F_1$ is a confluent hypergeometric function. From this derivation of the normalising constant, one deduces that the mean only exists for $\alpha>2$ (while the inverse normal distribution corresponds to $\alpha=2$) and the variance only exists for $\alpha>3$. The mean is given by $$\mathbb{E}[X]=\frac{\mu}{\sigma^2}\,\frac{{}_1F_1\left(\frac{1}{2}(\alpha-3);\frac{3}{2};\frac{\mu^2}{2\sigma^2}\right)}{{}_1F_1\left(\frac{1}{2}(\alpha-1);\frac{1}{2};\frac{\mu^2}{2\sigma^2}\right)}$$

A much simpler argument as to why the mean does not exist is that, if $f$ is the normal $\text{N}(b,p)$ density, $$x^{-1} f(x)\equiv \frac{e^{-b^2/2p}}{\sqrt{2\pi p}}x^{-1}$$ at zero, which is not integrable.

• @Xian Do you have a solution for the Cauchy principal value of $\mathbb{E}[X]$ when $\alpha=2$? Jul 22, 2017 at 19:55
• @AaronHendrickson: I do not see how this is connected with this question or my answer. But more to the point, I do not have a solution! Jul 22, 2017 at 20:35

This is equivalent to asking for $\int_{- \infty}^\infty \frac{1}{x} P(X=x) \, dx$ and $\int_{- \infty}^\infty \frac{1}{x^2} P(X=x) \, dx$.

Considering the second, let's choose some arbitrary $a > 0$: \eqalign { \int_{- \infty}^\infty \frac{1}{x^2} P(X=x) \, dx &= \int_{- \infty}^{-a} \frac{1}{x^2} P(X=x) \, dx\\ &+ \int_{-a}^a \frac{1}{x^2} P(X=x) \, dx + \int_a^\infty \frac{1}{x^2} P(X=x) \, dx \\ &\ge \int_{-a}^a \frac{1}{x^2} P(X=x) \, dx \\ &\ge \int_{-a}^a \frac{1}{x^2} \min\left(P(X=-a), P(X=a)\right) \, dx \\ &= \min\left(P(X=-a), P(X=a)\right) \int_{-a}^a \frac{1}{x^2} \, dx \to \infty } where first we split up the integral and note that the tails are nonnegative, and then use the fact that the normal pdf is quasiconcave. So $\mathbb{E}\left[\frac{1}{X^2}\right]$ doesn't exist.

We can do exactly the same thing to show that $\mathbb{E}\left[\frac{1}{\lvert X \rvert}\right]$ doesn't exist (since $\int_{0}^a \frac{1}{X} dx$ also diverges). But $\frac{1}{X}$ is a little trickier, since one of the tail integrals is negative-valued. Still, $\mathbb{E}[\frac{1}{X} \mid \frac{1}{X} < a]$ doesn't exist, since the integral diverges. I think that, because $P(\frac{1}{X} < a) > 0$, using basically the same argument as the law of total expectation, this then implies that the overall expectation cannot exist. But I'm not 100% on that.

• There is a cute application of quasiconcavity here. Some comments: (1) Essentially the same argument works with a much weaker hypothesis, though. As long as the density is continuous and nonzero at $x = 0$, that is already enough. This follows because a nonzero minimum is attained on $[-a,a]$ for some $a > 0$ and the rest falls out. (2) Because both $\mathbb E \max(X^{-1},0)$ and $\mathbb E -\min(X^{-1},0)$ are infinite, then $\mathbb E X^{-1}$ does not exist. (3) Strictly speaking, $\mathbb EX^{-2}$ does exist since the integrand is nonnegative, but it is not finite. (cont.) Sep 15, 2013 at 13:39
• (cont.) (4) It is clear that you are using the notation $\mathbb P(X = x)$ to refer to the density here, but a little caution might be advised. (+1) :-) Sep 15, 2013 at 13:41
• @cardinal 1) Yes, absolutely. 2-3) Okay; I've seen "does not exist" and "infinite" used interchangeably, e.g. I think in Feller, but the distinction makes sense. 4) Whoops, yes, that should have been a lowercase p -- it was very late at night when I wrote this. :) Sep 15, 2013 at 19:30
• In case I didn't state it very clearly, what I intended for point 2 was simply that you need not be less than 100% confident about the result you arrived at! Cheers. :-) Sep 16, 2013 at 0:48
• To second @cardinal's argument, I also think it is enough to prove that $\mathbb{E}[|X|^{-1}]$ does not exist (or more precisely is infinite) to conclude that $\mathbb{E}[X^{-1}]$ does not exist, by using the $|X|^-1= \max(X^{-1},0)-\min(X^{-1},0)$ decomposition. Jun 30, 2016 at 6:17

It is true that, as others have said, the means of $$Y$$ and $$Y^{2}$$ ($$Y=1/X$$, where $$X$$ is normally distributed with mean $$\mu$$ and standard deviation $$\sigma$$) don't exist as ordinary integrals. Nevertheless, the mean of $$Y$$ exists in the principal value sense, and can be expressed in terms of a well-understood special function, the Dawson function.

The mean of $$Y^{2}$$ also seems to exist in some generalized sense, and can also be expressed in terms of the Dawson function. But this generalized sense, whatever it is, is more convoluted than just the principal value.

The analytic expressions for the means of $$Y$$ and $$Y^{2}$$ can be compared to the results one gets from (pseudo-) random sampling from the normal distribution, and I will do that below. One draws the values of $$X$$ from the normal distribution, and computes the values $$1/X$$ and $$1/X^{2}$$. When one computes the average of all the $$1/X$$ values, one finds excellent agreement with the prediction of the analytic expression (as long as $$\mu$$ is not too close to zero). And similarly, when one computes the average of all the $$1/X^{2}$$ values, one finds excellent agreement with the prediction of the analytic expression (again, as long as $$\mu$$ is not too close to zero).

That the mean of $$Y$$ is expressible as a principal value integral is explained e.g. on Wikipedia, but I will summarize it here (and add a few extra bits of information). I will also argue that taking the principal value is entirely consistent with the interpretation of the integral as a probabilistic mean value.

As far as why the mean of $$Y^{2}$$ is also ultimately computable, I will unfortunately only be able to give a heuristic argument.

The analytic results

$$\text{P.V.}\;\mathbb{E}[Y]=\text{P.V.}\;\int_{-\infty}^{\infty}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx=\lim_{\epsilon\to 0^{+}}\left(\int_{-\infty}^{-\epsilon}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx+\int_{\epsilon}^{\infty}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx\right)$$ $$=\frac{\sqrt{2}}{\sigma}F\left(\frac{\mu}{\sqrt{2}\,\sigma}\right)\,,$$ and $$\mathbb{E}[Y^2]\overset{\overset{\scriptstyle\text{some}}{\scriptstyle\text{sense?}}}{=}\frac{1}{\sigma^{2}}\left(\mu\,\mathbb{E}[Y]-1\right)=\frac{1}{\sigma^{2}}\left(\frac{\sqrt{2}\mu}{\sigma}F\left(\frac{\mu}{\sqrt{2}\,\sigma}\right)-1\right)=-\frac{1}{\sigma^{2}}F'\left(\frac{\mu}{\sqrt{2}\,\sigma}\right)\,.$$

In the expressions above, $$F(x)$$ is the Dawson function $$F(x)=e^{-x^{2}}\int_{0}^{x}e^{t^{2}}dt$$ for which efficient evaluation algorithms exist (it is for example implemented in Mathematica as DawsonF, and you can evaluate it on Wolfram Alpha, like so). Also, $$F'(x)$$ is its derivative, which satisfies $$F'(x)=1-2xF(x)$$. Among other useful properties are that $$F(-x)=-F(x)$$ and $$F'(-x)=F'(x)$$.

Series expansions

One can also use the following series expansions:

$$\text{P.V.}\;\mathbb{E}[Y]=\frac{1}{\mu}+\frac{\sigma^{2}}{\mu^{3}}+\frac{3\sigma^{4}}{\mu^{5}}+\frac{15 \sigma^{6}}{\mu^{7}}+\frac{105\sigma^{8}}{\mu^{9}}+\frac{945 \sigma^{10}}{\mu^{11}}+\cdots\hspace{2.5em}$$ $$\mathbb{E}[Y^{2}]\overset{\overset{\scriptstyle\text{some}}{\scriptstyle\text{sense?}}}{=}\frac{1}{\mu^{2}}+\frac{3\sigma^{2}}{\mu^{4}}+\frac{15\sigma^{4}}{\mu^{6}}+\frac{105 \sigma^{6}}{\mu^{8}}+\frac{945\sigma^{8}}{\mu^{10}}+\frac{10395 \sigma^{10}}{\mu^{12}}+\cdots$$

Comparison to the results from random sampling

All the results above can be tested (e.g. in Mathematica) by comparing them to the mean values computed from (pseudo-) random samples. For $$\mu=0.9$$ and $$\sigma=0.1$$, with $$N=10^{8}$$ sample points, I get the following results:

$$\mathbb{E}[Y]$$                $$\mathbb{E}[Y^{2}]$$
analytic                 1.125 371 01      1.283 390 59
series                   1.125 370 96      1.283 389 86
random sample    1.125 377 74      1.283 410 66

As far as the integral defining $$\mathbb{E}[Y]$$, at this point, I would like to argue that it makes sense that we take that integral in the principal value sense. In a nutshell, the mean value computed from a random sample is precisely what we would get if we performed Monte Carlo integration of the relevant integral, provided that the integral is taken in the principal value sense. This is because the Monte Carlo sampling does not proceed first separately below the singularity, then separately above. It is rather constantly sampling both above and below the singularity.

To put it another way, recall that taking the principal value makes it so that the divergence as one approaches the singularity from the left exactly cancels the divergence as one approaches the singularity from the right. But taking random samples will accomplish the same thing, since the samples are, in an appropriate limiting sense, equally likely to come from either side of the singularity. In other words, we are about equally likely to randomly draw $$X$$ values that are very slightly above zero and those that are very slightly below zero; and so we are about equally likely to get values of $$Y=1/X$$ that are very large in magnitude and positive and those that are very large in magnitude and negative. I say 'about equally likely', but the larger the magnitudes, the closer become the two likelihoods (of drawing the negative and positive values of that magnitude). In this way, the mean value remains finite.

Failure when $$\mu$$ is too close to zero

Once $$\mu$$ starts to get too close to zero, one would of course expect something to go bad. Indeed, the results for random sampling start to fluctuate widely from sample to sample (even with samples of $$10^{8}$$ points). In my samples, for $$\sigma=0.1$$, this happens for $$\mathbb{E}[Y^{2}]$$ when $$|\mu|$$ is below about 0.5, and for $$\mathbb{E}[Y]$$ when $$|\mu|$$ is below about 0.2.

On the analytical side, one obvious thing to notice is that, for $$|\mu|$$ low enough, the value of $$-\frac{1}{\sigma^{2}}F'\left(\frac{\mu}{\sqrt{2}\,\sigma}\right)$$ (which is 'in some sense' equal to $$\mathbb{E}[Y^2]$$) turns negative; in particular, note that $$F'(0)=1$$. This is obviously non-sensical. Moreover, as $$|\mu|$$ approaches zero, the magnitude of $$\text{P.V.}\;\mathbb{E}[Y]=\frac{\sqrt{2}}{\sigma}F\left(\frac{\mu}{\sqrt{2}\,\sigma}\right)$$ reaches a maximum and then decreases, reaching zero at $$\mu=0$$; this, too, is non-sensical.

So the analytical results are letting us know that they can't be trusted when $$|\mu|$$ is too small, but they don't tell us how small a $$|\mu|$$, precisely, is too small. They also don't tell us what the nature of the failure is, exactly. Finally, the derivation of the analytical results doesn't offer (at least to me) any reason to predict that small values of $$|\mu|$$ are in any way dangerous.

Derivation of the analytic formulas

The mean value of $$Y$$

By changing the variables of integration, we have that $$\text{P.V.}\;\int_{-\infty}^{\infty}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx=-\frac{1}{\sqrt{2\pi}\,\sigma}\;\left(\text{P.V.}\;\int_{-\infty}^{\infty}\frac{1}{-\frac{\mu}{\sqrt{2}\sigma}-y}e^{-y^{2}}dy\right)$$ $$=-\frac{\pi}{\sqrt{2\pi}\,\sigma}\;H\left(-\frac{\mu}{\sqrt{2}\sigma}\right)\,,$$ where $$H(z)$$ is is the Hilbert transform of a Gaussian: $$H(z)=\frac{1}{\pi}\;\text{P.V.}\;\int_{-\infty}^{\infty}\frac{e^{-x^{2}}}{z-x}dx\,.$$ The Hilbert transform of a Gaussian, in turn, is expressible in terms of the Dawson function (for a derivation, see e.g. here, or this paper): $$H(z)=\frac{2}{\sqrt{\pi}}F(z)\,.$$ The result is the formula given above.

The mean value of $$Y^{2}$$

This is trickier, and I can only offer a heuristic derivation. Notice that, formally speaking, the integral $$\int_{-\infty}^{\infty}\frac{1}{x^{2}}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx$$ doesn't exist even in the principal value sense: the divergence around zero is positive both below and above zero, so there is no way for the two infinite areas to cancel.

Heuristically, however, we can proceed as follows. We begin with the following change of variables, which is nonproblematic:

$$\text{P.V.}\;\int_{-\infty}^{\infty}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx = \text{P.V.}\;\int_{-\infty}^{\infty}\frac{1}{u+\mu}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{u^{2}}{2\sigma^{2}}}du$$

Now we note that a formal differentiation of the two sides of this equality with respect to $$\mu$$ gives, on the right-hand side, $$\frac{d}{d\mu} \int_{-\infty}^{\infty}\frac{1}{u+\mu}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{u^{2}}{2\sigma^{2}}}du=-\int_{-\infty}^{\infty}\frac{1}{(u+\mu)^{2}}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{u^{2}}{2\sigma^{2}}}du$$ $$=-\int_{-\infty}^{\infty}\frac{1}{x^{2}}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx\,.$$

This last expression (if the integral existed!) would be $$-\mathbb{E}[Y^{2}]$$.

And on the left-hand side, a formal differentiation gives $$\frac{d}{d\mu}\int_{-\infty}^{\infty}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx=\frac{1}{\sigma^{2}}\left(1-\mu\int_{-\infty}^{\infty}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx\right).$$

The last expression becomes $$\frac{1}{\sigma^{2}}\left(1-\mu\,\mathbb{E}[Y]\right)$$, provided we (all of the sudden) again take the integral in the principal value sense. In this heuristic way, we get that $$\mathbb{E}[Y^{2}]=\frac{1}{\sigma^{2}}\left(\mu\,\mathbb{E}[Y]-1\right)\,.$$

Just to illustrate some of the difficulties ignored in this heuristic approach, let us consider what happens if we try to treat the principal values honestly. Then, when we differentiate the right-hand side above, we must write

$$\frac{d}{d\mu} \lim_{\epsilon\to 0^{+}} \left(\int_{-\infty}^{-\mu-\epsilon}\frac{1}{u+\mu}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{u^{2}}{2\sigma^{2}}}du+\int_{-\mu+\epsilon}^{\infty}\frac{1}{u+\mu}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{u^{2}}{2\sigma^{2}}}du\right).$$

Let's assume that we can move the differentiation inside the limit (I don't have a proof that this is OK, but probably it actually is). We get $$\lim_{\epsilon\to 0^{+}} \left[\frac{1}{\epsilon}\frac{1}{\sqrt{2\pi}\sigma}\left(e^{-\frac{(\mu+\epsilon)^{2}}{2\sigma^{2}}}+e^{-\frac{(\mu-\epsilon)^{2}}{2\sigma^{2}}}\right)\right.$$ $$\left.-\int_{-\infty}^{-\mu-\epsilon}\frac{1}{(u+\mu)^{2}}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{u^{2}}{2\sigma^{2}}}du-\int_{-\mu+\epsilon}^{\infty}\frac{1}{(u+\mu)^{2}}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{u^{2}}{2\sigma^{2}}}du\right]\,.$$ The first term, which comes from the differentiation of the limits of integration, diverges (when $$\epsilon\to 0$$) as $$\frac{1}{\epsilon}\frac{2}{\sqrt{2\pi}\sigma}e^{-\frac{\mu^{2}}{2\sigma^{2}}}$$, and presumably exactly cancels the divergences from the two integrals that follow. (Borrowing terminology from particle physics, this divergence-cancelling term might be called a 'counterterm'.) So the limit as a whole may well exist.

However, it is not clear why this rather complicated 'limit as a whole' should be identified with $$-\mathbb{E}[Y^{2}]$$. True, the two integrals are suggestive of it (once we change variables to $$x=u+\mu$$), but we have no interpretation for the counterterm.

So, our heuristic argument indeed ignored some serious difficulties. Nevertheless, as we have seen above, there is something right about all this, because the analytic result for $$\mathbb{E}[Y^{2}]$$ agrees extremely well with the results obtained from random samples (as long as $$\mu$$ is not too close to zero).

Derivation of series expansions

A simple change of variables gives

$$\text{P.V.}\;\mathbb{E}[Y]=\text{P.V.}\;\int_{-\infty}^{\infty}\frac{1}{x}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx=\text{P.V.}\;\int_{-\infty}^{\infty}\frac{1}{\mu+y}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{y^{2}}{2\sigma^{2}}}dy$$

We use the series expansion

$$\frac{1}{\mu+y}=\frac{1}{\mu}-\frac{y}{\mu^2}+\frac{y^2}{\mu^3}-\frac{y^3}{\mu^4}+\cdots$$

and integrate term by term. The result is given above.

For the series expansion of $$\mathbb{E}[Y^{2}]$$, we start with a similar (but merely 'formal') expression $$\mathbb{E}[Y^{2}]=\int_{-\infty}^{\infty}\frac{1}{(\mu+y)^{2}}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{y^{2}}{2\sigma^{2}}}dy\,.$$ As we've noted bove, this integral doesn't exist even in the principal value sense; it requires a 'counterterm'. Nevertheless, inside the integral, we insert a series expansion of $$1/(\mu+y)^{2}$$, and formally integrate term by term. We obtain the result quoted above. And as we've seem, the resulting expansion works extremely well (when compared to both the analytic result and the result from random samples), as long as $$\mu$$ is not too close to zero.

• This looks interesting, thank you. However, I stopped reading closely about a quarter of the way in after noting that even a huge Monte-Carlo sample from $N(0.9, 0.1)$ will almost never sample even a single negative value. (You would need to draw around $10^{19}$ values before seeing a negative number.) Thus, you are effectively truncating these distributions and there consequently is no mystery why any of the expectations will exist and be finite. I think this might have useful applications, but that possibility seems buried under what ultimately looks like superfluous mathematics.
– whuber
Jul 26, 2021 at 17:51
• @whuber Yes, but that raises the following question: what, precisely, is the correct probability-theory interpretation of the 'analytic formula for' E(Y^2)? Equivalently, what is the probability-theory interpretation of the 'counterterm'? And also it raises this practical question: suppose we will be taking samples of size N. Suppose the sigma (the standard deviation) of the normal distribution is given. How small can the mu (the mean) of the normal distribution get before we have to start to worry that E(Y^2) will start to become unpredictable? (Or, for a given mu, how large an N?) Jul 27, 2021 at 18:40
• Those are good questions. One can answer them by studying the Normal distribution, using calculations similar to the one I used in my first comment.
– whuber
Jul 27, 2021 at 18:43