95% Confidence interval of $\lambda$ for $X_1,...,X_n$ IID exponential with rate $\lambda$

Question

I know how how to find the estimation of $\hat{\lambda}$ using the method of moments. I can take the first moment and equate it to the empirical to get,

$E(X) = \frac{1}{\lambda} = \frac{\sum_{i=1}^{n}{x_i}}{n}=\bar{x}$.

This gives $\hat{\lambda}=\frac{1}{\bar{x}}$.

Using the delta method to find $var(\hat{\lambda})$, the result comes out to be $var(\hat{\lambda}) = \frac{\lambda^2}{n}$

My question is how can I obtain a 95% confidence interval for $\lambda$ based on a sample $(x_i, . . . ,x_n)$, when $n$ is large?

My thought is since we are trying to estimate $\lambda$, how can we obtain a confidence interval for $\lambda$? Do I from the confidence interval with $\hat\lambda$? From doing that I get,

$\hat\lambda$ $\pm$ $z_\frac{\alpha}{2}\sqrt{var(\hat\lambda)}$ $\Longrightarrow$ $\hat\lambda$ $\pm$ $z_\frac{\alpha}{2}\sqrt{\frac{\lambda^2}{n}}$ $\Longrightarrow$ $\hat\lambda$ $\pm$ $1.96\sqrt{\frac{\lambda^2}{n}}$

How can the confidence interval of $\lambda$ have $\hat\lambda$ and $\lambda$ in the confidence interval?

Answer 1

There are many different forms of confidence intervals you could use here. In my view, the simplest would be to use the central limit theorem form a probability statement for the difference between the sample mean and the true mean, and then "invert" this to get a corresponding statement for the parameter $\lambda$.

Since the data come from an exponential distribution, the variance is the square of the mean. Thus, applying the central limit theorem, for large $n$, we have the approximate distribution $\bar{X} \sim \text{N}(\mu, \mu^2 /n)$, where $\mu = 1 / \lambda$. This gives us the approximate probability interval:

$$\begin{equation} \begin{aligned} \alpha &\approx \mathbb{P} \Bigg( - z_{\alpha/2} \leqslant \frac{\bar{X} - \mu}{\mu / \sqrt{n}} \leqslant z_{\alpha/2} \Bigg) \\[6pt] &= \mathbb{P} \Bigg( - z_{\alpha/2} \leqslant \sqrt{n} \cdot \frac{1 / \hat{\lambda} - 1/ \lambda }{1 / \lambda} \leqslant z_{\alpha/2} \Bigg) \\[6pt] &= \mathbb{P} \Bigg( - z_{\alpha/2} \leqslant \sqrt{n} \cdot \Big( \frac{\lambda}{\hat{\lambda}} - 1 \Big) \leqslant z_{\alpha/2} \Bigg) \\[6pt] &= \mathbb{P} \Bigg( 1 - \frac{z_{\alpha/2}}{\sqrt{n}} \leqslant \frac{\lambda}{\hat{\lambda}} \leqslant 1 + \frac{z_{\alpha/2}}{\sqrt{n}} \Bigg) \\[6pt] &= \mathbb{P} \Bigg( \hat{\lambda} \Big( 1 - \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \leqslant \lambda \leqslant \hat{\lambda} \Big( 1 + \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \Bigg) \\[6pt] &= \mathbb{P} \Bigg( \frac{1}{\bar{X}} \Big( 1 - \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \leqslant \lambda \leqslant \frac{1}{\bar{X}} \Big( 1 + \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \Bigg). \\[6pt] \end{aligned} \end{equation}$$

Substituting the sample data leads to the confidence interval:

$$\text{CI}_\lambda(1-\alpha) \equiv \Bigg[ \frac{1}{\bar{x}} \Big( 1 - \frac{z_{\alpha/2}}{\sqrt{n}} \Big) , \frac{1}{\bar{x}} \Big( 1 + \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \Bigg].$$

(Note: If $n < z_{\alpha/2}^2$ then the lower bound for this confidence interval will be below zero. You should not use this confidence interval form if this is the case.) Note also from the other answer by BruceET, that you can use the exact distribution of the sample mean (the gamma distribution) to remove the approximation. (The gamma approaches the normal asymptotically, so this wont make much difference when $n$ is large.)

Answer 2

If you have a random sample of size $n$ from $\mathsf{Exp}(\lambda = 1/\mu),$ then $\bar X \sim \mathsf{Gamma}(n,\, \text{rate}=n\lambda).$

Thus, $P(L \le \bar X\lambda = \bar X/\mu \le U)=0.95,$ where $L$ and $U$ cut probability 0.025 from the lower and upper tails of $\mathsf{Gamma}(\text{shape} = n,\, \text{rate}=n),$ respectively.

This implies that $P(\bar X/U \le \mu < \bar X/L) = .95,$ so that a 95% CI for $\mu$ is of the form $(\bar X/U, \bar X/L).$

For example, if $\mu = 1/\lambda = 5$ and $n = 10,$ we might get $\bar X = 3.80.$ Then a 95% CI for $\mu$ is $(2.22, 7.92).$ Notice that $\bar X$ is contained in this CI, but the sample mean does not lie at the center of the CI. Computations in R:

set.seed(1234); a = mean(rexp(10, 1/5)); a
[1] 3.800074
a/qgamma(c(.975,.025), 10, 10)
[1] 2.224242 7.924434

Notes: (1) The Wikipedia article on exponential distributions discusses inference in some detail; under 'Confidence Interval' the article has a confidence interval equivalent to the one shown above, but in terms of a chi-squared distribution. (This makes it possible to find confidence limits using printed chi-squared tables.)

(2) To get an exact 95% CI for rate $\lambda,$ take reciprocals: $(L/\bar X,\, U/\bar X).$ However, notice that $1/\bar X$ is a biased point estimator of $\lambda,$ with bias becoming negligible for large $n.$

(3) Because the exponential distribution is highly skewed it is inappropriate to use a symmetrical CI of the form $\bar X \pm M,$ were $M$ is a margin of error based on a (symmetrical) normal distribution, unless the sample size is sufficiently large. For sufficiently large $n,$ the mean $\bar X$ of an exponential sample becomes approximately normal, and a symmetrical CI is a reasonable approximation.

(4) Below is a simulation of a million $\bar X$'s based on random samples of size $n = 10$ from $\mathsf{Exp}(\mu = 5).$ The histogram illustrates that $Q = \bar X/\mu \sim \mathsf{Gamma}(\text{shape}=n, \text{rate}=n),$ as claimed above. A formal proof uses moment generating functions.

set.seed(2019)
a = replicate(10^6, mean(rexp(10, 1/5)))
hist(a/5, prob=T, col="skyblue2", xlab="Q", 
     main="GAMMA(10,10)")
  curve(dgamma(x,10,10), add=T, lwd=2)
  abline(v = qgamma(c(.025,.975), 10, 10), 
     lwd=2, col="red", lty="dashed")