Confidence interval for exponential distribution

Question

Let $X_1,...,X_n$ random sample of $X$~$exp(\theta)$.

i) Find a exact confidence interval for $\theta$ with coefficient of confidence equal to $\gamma$

ii)Find a asymptotic confidence interval for $\theta$, with coefficient of confidence approximately $\gamma$

What I did

i)Let $Q(X;\theta)=2\theta\sum X_i$~$\chi^2_{2n}$ then $P(q_1\leq Q(X;\theta)\leq q_2)=\gamma$, that $q_1$ and $q_2$ are founded from values of the chi-square distribution. $$P(q_1\leq 2\theta\sum X_i\leq q_2)=P(\frac{q_1}{\sum X_i}\leq\theta\leq\frac{q_2}{\sum X_i})=\gamma$$ then $IC[\gamma;\theta]=[\frac{q_1}{\sum X_i};\frac{q_2}{\sum X_i}]$

ii) Here I am a little lost on how to proceed, I have to try to approach by the normal using delta or something method?

EDIT: If $\overline{X}$~$N(\frac{1}{\theta},\frac{1}{n\theta^2})$ if I take $\theta\overline{X}$ $$E[\theta\overline{X}]=\theta E[\frac{1}{n}\sum X_i]=\theta E[X_1]=\theta\frac{1}{\theta}=1$$ $$Var(\theta\overline{X})=\theta^2 Var(\overline{X})=\theta^2 Var(\frac{1}{n}\sum X_i)=\frac{\theta^2}{n^2}Var(\sum X_i)=\frac{\theta^2}{n}Var(X_1)=\frac{1}{n}$$

It's a $N(1,\frac{1}{n})$?

That's a guess not know if I can do this and not if it's right, waiting for someone more experienced.

Answer 1

The asymptotic confidence interval may be based on the (asymptotic) distribution of the mle. The Fisher information for this problem is given by $\frac{1}{\theta^2}$. Hence an asymptotic CI for $\theta$ is given by

$$\bar{X} \pm 1.96 \sqrt{\frac{\bar{X}^2}{n}}$$

where we have replaced $\theta^2$ by its mle, since we do not know the population parameter.

And here is a very simple R-simulation of the coverage for the case of a sample of size fifty from an exponential distribution with parameter $2$.

r<-rep(0,1000)
for(i in 1:1000){
  x<-rexp(50,2)
mle<-mean(x)
if(1/2<=mle+qnorm(0.975)*sqrt((mle^2)/50) & 1/2>=mle+qnorm(0.025)*sqrt((mle^2)/50)){r[i]<-1}
}
sum(r==1)
 [1] 948

Answer 2

For ii) you have several options but two of them are particularly appealing. One option is to go for a Wald-type confidence interval the other is to go with an interval based on the log-likelihood ratio statistic.

For the Wald-type you first get the limiting distribution of the MLE of $\theta$, thus something along the line $\hat \theta \sim N(\theta, \text{se}(\hat\theta)^2)$, where $\text{se}(\hat\theta)^2$ is the estimated variance of the distribution of $\hat\theta$, the MLE of $\theta$. The Wald-type interval of approximate confidence level $1-\alpha$ is (the usual)

$$ \hat\theta \pm z_{\alpha/2}\text{se}(\hat\theta). $$ where $z_{\alpha/2}$ is s.t. $P(Z\leq z_{\alpha/2}) = \alpha/2$ and $Z\sim N(0,1)$. This is what JohnK is suggesting in his answer.

For the confidence interval based on the log-likelihood ratio statistic, you find the full explanation here with the associated R code. You'll have to adapt it to your situation; i.e. just replace the log-likelihood function used there with yours. Note that with this method you'll have to use a computer to compute the interval since the method entails the inversion of a non-linear function.

These two confidence intervals are approximate in that the coverage probability for a given fixed sample will not typically be exactly $1-\alpha$; nor will it be always $\geq 1-\alpha$. But, the guarantee is that for increasing sample size, we expect the coverage probability to converge at $1-\alpha$.

Answer 3

I will explain a bit more deeply the process @JohnK gone through:

$$\hat\lambda_\text{MLE} = \frac{1}{\bar{X}}$$

$$l''=\log(L)'' = \frac{-n}{\lambda^2}$$

Therefore, $-E[l''] =$ Fisher's Information $= \frac{1}{\lambda^2}$. ($n=1$ for a single sample variance)

Now: $$I(\hat{\lambda}_\text{MLE}) = \bar{X}^2$$

The variance of the estimator: $$\frac{1}{I({\lambda})n } = \frac{1}{\frac{n}{\lambda^2}}=\frac{1}{\frac{n}{\hat{\lambda}_\text{MLE}^2}} = \frac{\bar{X}^2}{n}$$