# Finding expectation of reciprocal of sample mean

Consider the following distribution belonging to the exponential family.

$p_{\theta}(x) = \theta e^{-\theta x}$

The MLE estimating the $\theta$ parameter is

$\newcommand{\thetaMLE}{\hat{\theta}_{\mathrm{MLE}}}\thetaMLE = 1/\bar{X}$

where $\bar{X}$ is the sample mean.

How do we find the bias of this estimator?

$\mathrm{bias}(\thetaMLE) = E[\thetaMLE] - \theta = E[1/\bar{X}] - \theta$

How do we find expectation of the reciprocal of the sample mean?

• Is this homework? If so, please add the homework tag. – cardinal Oct 9 '11 at 1:28
• Hint: $1/\bar{X} = n / S_n$ where $S_n = \sum_{k=1}^n X_k$. Now, what is the distribution of $S_n$? From that, can you find $\mathbb E (1/S_n)$? Use linearity to conclude. – cardinal Oct 9 '11 at 1:28
• No this is not homework. Thought of this question while studying for exam. – Rohit Banga Oct 9 '11 at 2:24
• This is a pretty standard example/exercise found in most intro math stats books. What text are you using for the class? – cardinal Oct 9 '11 at 17:19
• Mostly lecture notes – Rohit Banga Oct 9 '11 at 18:42

First you find the distribution of the sample mean. The easiest way to do this is to use moment generating function. For exponential distribution, we have

$$m_{X_i}(t)=E\left[\exp\left(tX_i\right)\right]=\left(1-\frac{t}{\theta}\right)^{-1}$$

For sample mean we have

$$m_{\overline{X}}(t)=E\left[\exp\left(t \overline{X}\right)\right]=E\left[\exp\left(tN^{-1}\sum_{i=1}^{N}X_i\right)\right]=E\left[\prod_{i=1}^{N}\exp(tN^{-1}X_{i})\right]$$

Because of independence, we can interchange the product and expectation operations. so we get.

$$m_{\overline{X}}(t)=\prod_{i=1}^{N}E\left[\exp(tN^{-1}X_{i})\right]=\prod_{i=1}^{N}m_{X_i}(tN^{-1})=\left(1-\frac{t}{N\theta}\right)^{-N}$$

This is the moment generating function of a gamma distribution with shape parameter $N$ and inverse scale parameter $N\theta$, which has mean value of $\frac{1}{\theta}$, showing that the MLE is unbiased for the parameter $\beta=\frac{1}{\theta}$.

Now we simply take the expected value of $\frac{1}{\overline{X}}$ where $\overline{X}\sim Gamma(N,N\theta)$, which is given by:

$$E\left(\frac{1}{\overline{X}}\right)=\int_0^{\infty}\frac{1}{\overline{X}}f(\overline{X})d\overline{X}=\int_0^{\infty}\frac{1}{\overline{X}}\frac{(N\theta)^N \overline{X}^{N-1}\exp(-N\theta \overline{X})}{\Gamma(N)}d\overline{X}$$

In the integral, make the change of variables $t=N\theta \overline{X}\implies dt=N\theta d\overline{X}$, and we get

$$E\left(\frac{1}{\overline{X}}\right)=\frac{1}{\Gamma(N)}\int_0^{\infty}\frac{N\theta}{t}t^{N-1}\exp(-t)dt$$

$$=\frac{N\theta}{\Gamma(N)}\int_0^{\infty}t^{N-2}\exp(-t)dt=\frac{N\theta\Gamma(N-1)}{\Gamma(N)}=\frac{N\theta}{N-1}$$

Provided that $N\neq 1$, otherwise the expectation does not exist (and hence niether does the bias). So you have a bias of:

$$E\left(\frac{1}{\overline{X}}-\theta\right)=\frac{\theta}{N-1}$$

• That seems to be a nice solution. I would not have thought on the lines of mgf and gamma distributions given my noob status. – Rohit Banga Oct 9 '11 at 2:49