Finding expectation of reciprocal of sample mean Consider the following distribution belonging to the exponential family.
$p_{\theta}(x) = \theta e^{-\theta x} $
The MLE estimating the $\theta$ parameter is
$\newcommand{\thetaMLE}{\hat{\theta}_{\mathrm{MLE}}}\thetaMLE = 1/\bar{X}$
where $\bar{X}$ is the sample mean.
How do we find the bias of this estimator?
$\mathrm{bias}(\thetaMLE) = E[\thetaMLE] - \theta = E[1/\bar{X}] - \theta$
How do we find expectation of the reciprocal of the sample mean?
 A: First you find the distribution of the sample mean.  The easiest way to do this is to use moment generating function.  For exponential distribution, we have
$$m_{X_i}(t)=E\left[\exp\left(tX_i\right)\right]=\left(1-\frac{t}{\theta}\right)^{-1}$$
For sample mean we have
$$m_{\overline{X}}(t)=E\left[\exp\left(t \overline{X}\right)\right]=E\left[\exp\left(tN^{-1}\sum_{i=1}^{N}X_i\right)\right]=E\left[\prod_{i=1}^{N}\exp(tN^{-1}X_{i})\right]$$
Because of independence, we can interchange the product and expectation operations. so we get. 
$$m_{\overline{X}}(t)=\prod_{i=1}^{N}E\left[\exp(tN^{-1}X_{i})\right]=\prod_{i=1}^{N}m_{X_i}(tN^{-1})=\left(1-\frac{t}{N\theta}\right)^{-N}$$
This is the moment generating function of a gamma distribution with shape parameter $N$ and inverse scale parameter $N\theta$, which has mean value of $\frac{1}{\theta}$, showing that the MLE is unbiased for the parameter $\beta=\frac{1}{\theta}$.
Now we simply take the expected value of $\frac{1}{\overline{X}}$ where $\overline{X}\sim Gamma(N,N\theta)$, which is given by:
$$E\left(\frac{1}{\overline{X}}\right)=\int_0^{\infty}\frac{1}{\overline{X}}f(\overline{X})d\overline{X}=\int_0^{\infty}\frac{1}{\overline{X}}\frac{(N\theta)^N \overline{X}^{N-1}\exp(-N\theta \overline{X})}{\Gamma(N)}d\overline{X}$$
In the integral, make the change of variables $t=N\theta \overline{X}\implies dt=N\theta d\overline{X}$, and we get
$$E\left(\frac{1}{\overline{X}}\right)=\frac{1}{\Gamma(N)}\int_0^{\infty}\frac{N\theta}{t}t^{N-1}\exp(-t)dt$$
$$=\frac{N\theta}{\Gamma(N)}\int_0^{\infty}t^{N-2}\exp(-t)dt=\frac{N\theta\Gamma(N-1)}{\Gamma(N)}=\frac{N\theta}{N-1}$$
Provided that $N\neq 1$, otherwise the expectation does not exist (and hence niether does the bias).  So you have a bias of:
$$E\left(\frac{1}{\overline{X}}-\theta\right)=\frac{\theta}{N-1}$$
