# 95% Confidence interval of $\lambda$ for $X_1,…,X_n$ IID exponential with rate $\lambda$

I know how how to find the estimation of $$\hat{\lambda}$$ using the method of moments. I can take the first moment and equate it to the empirical to get,

$$E(X) = \frac{1}{\lambda} = \frac{\sum_{i=1}^{n}{x_i}}{n}=\bar{x}$$.

This gives $$\hat{\lambda}=\frac{1}{\bar{x}}$$.

Using the delta method to find $$var(\hat{\lambda})$$, the result comes out to be $$var(\hat{\lambda}) = \frac{\lambda^2}{n}$$

My question is how can I obtain a 95% confidence interval for $$\lambda$$ based on a sample $$(x_i, . . . ,x_n)$$, when $$n$$ is large?

My thought is since we are trying to estimate $$\lambda$$, how can we obtain a confidence interval for $$\lambda$$? Do I from the confidence interval with $$\hat\lambda$$? From doing that I get,

$$\hat\lambda$$ $$\pm$$ $$z_\frac{\alpha}{2}\sqrt{var(\hat\lambda)}$$ $$\Longrightarrow$$ $$\hat\lambda$$ $$\pm$$ $$z_\frac{\alpha}{2}\sqrt{\frac{\lambda^2}{n}}$$ $$\Longrightarrow$$ $$\hat\lambda$$ $$\pm$$ $$1.96\sqrt{\frac{\lambda^2}{n}}$$

How can the confidence interval of $$\lambda$$ have $$\hat\lambda$$ and $$\lambda$$ in the confidence interval?

• Simple (very-large-n) version: You would need to substitute your estimate of lambda into the second term. More complex, smaller-but-still-large-n version: you can write an inequality in terms of both $\lambda$ and $\hat{\lambda}$ and attempt to "solve"/back out endpoints on $\lambda$. By comparison see the Wikipedia pag on CIs for the binomial proportion and compare the normal interval with the Wilson score interval to see an example of the two notions – Glen_b -Reinstate Monica Mar 28 '19 at 0:29
• So I have to substitute $\hat\lambda$ for the $\lambda$ value inside the square root of the confidence interval? Is it correct if the confidence interval has terms of $\bar{x}$ instead of $\hat\lambda$? – Wade Mar 28 '19 at 0:35
• Since $\hat{\lambda}$ is a monotonic function of $\bar{x}$, converting from one to the other is simple. Two things to note: 1. The first interval I mentioned relies on both the CLT and Slutsky's theorem. The second approach is essentially the same as the one Ben did, while Bruce did something else again. All three are fine at large n, but Ben's works at a smaller n than the first one and Bruce's works at any n. – Glen_b -Reinstate Monica Mar 28 '19 at 2:15

There are many different forms of confidence intervals you could use here. In my view, the simplest would be to use the central limit theorem form a probability statement for the difference between the sample mean and the true mean, and then "invert" this to get a corresponding statement for the parameter $$\lambda$$.

Since the data come from an exponential distribution, the variance is the square of the mean. Thus, applying the central limit theorem, for large $$n$$, we have the approximate distribution $$\bar{X} \sim \text{N}(\mu, \mu^2 /n)$$, where $$\mu = 1 / \lambda$$. This gives us the approximate probability interval:

\begin{equation} \begin{aligned} \alpha &\approx \mathbb{P} \Bigg( - z_{\alpha/2} \leqslant \frac{\bar{X} - \mu}{\mu / \sqrt{n}} \leqslant z_{\alpha/2} \Bigg) \\[6pt] &= \mathbb{P} \Bigg( - z_{\alpha/2} \leqslant \sqrt{n} \cdot \frac{1 / \hat{\lambda} - 1/ \lambda }{1 / \lambda} \leqslant z_{\alpha/2} \Bigg) \\[6pt] &= \mathbb{P} \Bigg( - z_{\alpha/2} \leqslant \sqrt{n} \cdot \Big( \frac{\lambda}{\hat{\lambda}} - 1 \Big) \leqslant z_{\alpha/2} \Bigg) \\[6pt] &= \mathbb{P} \Bigg( 1 - \frac{z_{\alpha/2}}{\sqrt{n}} \leqslant \frac{\lambda}{\hat{\lambda}} \leqslant 1 + \frac{z_{\alpha/2}}{\sqrt{n}} \Bigg) \\[6pt] &= \mathbb{P} \Bigg( \hat{\lambda} \Big( 1 - \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \leqslant \lambda \leqslant \hat{\lambda} \Big( 1 + \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \Bigg) \\[6pt] &= \mathbb{P} \Bigg( \frac{1}{\bar{X}} \Big( 1 - \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \leqslant \lambda \leqslant \frac{1}{\bar{X}} \Big( 1 + \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \Bigg). \\[6pt] \end{aligned} \end{equation}

Substituting the sample data leads to the confidence interval:

$$\text{CI}_\lambda(1-\alpha) \equiv \Bigg[ \frac{1}{\bar{x}} \Big( 1 - \frac{z_{\alpha/2}}{\sqrt{n}} \Big) , \frac{1}{\bar{x}} \Big( 1 + \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \Bigg].$$

(Note: If $$n < z_{\alpha/2}^2$$ then the lower bound for this confidence interval will be below zero. You should not use this confidence interval form if this is the case.) Note also from the other answer by BruceET, that you can use the exact distribution of the sample mean (the gamma distribution) to remove the approximation. (The gamma approaches the normal asymptotically, so this wont make much difference when $$n$$ is large.)

• @Ben The confidence intervals you have are asymptotic i.e. $n$ must be quite large. I see that you say approximate intervals but then I believe it is a stretch to say that $n$ can be smaller than $z_{\alpha/2}$, what do you think? :-) – Math-fun Mar 28 '19 at 8:17
• @Math-fun: I agree - I am saying that if $n < z_{\alpha/2}^2$ then that is bad (i.e., do not use this confidence interval for such low values of $n$). – Ben - Reinstate Monica Mar 28 '19 at 9:04

If you have a random sample of size $$n$$ from $$\mathsf{Exp}(\lambda = 1/\mu),$$ then $$\bar X \sim \mathsf{Gamma}(n,\, \text{rate}=n\lambda).$$

Thus, $$P(L \le \bar X\lambda = \bar X/\mu \le U)=0.95,$$ where $$L$$ and $$U$$ cut probability 0.025 from the lower and upper tails of $$\mathsf{Gamma}(\text{shape} = n,\, \text{rate}=n),$$ respectively.

This implies that $$P(\bar X/U \le \mu < \bar X/L) = .95,$$ so that a 95% CI for $$\mu$$ is of the form $$(\bar X/U, \bar X/L).$$

For example, if $$\mu = 1/\lambda = 5$$ and $$n = 10,$$ we might get $$\bar X = 3.80.$$ Then a 95% CI for $$\mu$$ is $$(2.22, 7.92).$$ Notice that $$\bar X$$ is contained in this CI, but the sample mean does not lie at the center of the CI. Computations in R:

set.seed(1234); a = mean(rexp(10, 1/5)); a
 3.800074
a/qgamma(c(.975,.025), 10, 10)
 2.224242 7.924434


Notes: (1) The Wikipedia article on exponential distributions discusses inference in some detail; under 'Confidence Interval' the article has a confidence interval equivalent to the one shown above, but in terms of a chi-squared distribution. (This makes it possible to find confidence limits using printed chi-squared tables.)

(2) To get an exact 95% CI for rate $$\lambda,$$ take reciprocals: $$(L/\bar X,\, U/\bar X).$$ However, notice that $$1/\bar X$$ is a biased point estimator of $$\lambda,$$ with bias becoming negligible for large $$n.$$

(3) Because the exponential distribution is highly skewed it is inappropriate to use a symmetrical CI of the form $$\bar X \pm M,$$ were $$M$$ is a margin of error based on a (symmetrical) normal distribution, unless the sample size is sufficiently large. For sufficiently large $$n,$$ the mean $$\bar X$$ of an exponential sample becomes approximately normal, and a symmetrical CI is a reasonable approximation.

(4) Below is a simulation of a million $$\bar X$$'s based on random samples of size $$n = 10$$ from $$\mathsf{Exp}(\mu = 5).$$ The histogram illustrates that $$Q = \bar X/\mu \sim \mathsf{Gamma}(\text{shape}=n, \text{rate}=n),$$ as claimed above. A formal proof uses moment generating functions.

set.seed(2019)
a = replicate(10^6, mean(rexp(10, 1/5)))
hist(a/5, prob=T, col="skyblue2", xlab="Q",
main="GAMMA(10,10)") 