Maximum Likelihood estimator - confidence interval How can I construct an asymptotic confidence interval for a real parameter, starting from the MLE for that parameter?
 A: Use the fact that for an i.i.d. sample of size $n$, given some regularity conditions, the MLE $\hat{\theta}$ is a consistent estimator of the true parameter $\theta_0$, & its distribution asymptotically Normal, with variance determined by the reciprocal of the Fisher information:
$$\sqrt{n}\left(\hat{\theta}-\theta_0\right) \rightarrow \mathcal{N}\left(0,\frac{1}{\mathcal{I}_1(\theta_0)}\right)$$
where $\mathcal{I}_1(\theta_0)$ is the Fisher information from a single sample.
The observed information at the MLE $I(\hat{\theta})$ tends to the expected information asymptotically, so you can calculate (say 95%) confidence intervals with
$$\hat{\theta} \pm \frac{1.96}{\sqrt{nI_1\left(\hat\theta\right)}}$$
For example, if $X$ is a zero-truncated Poisson variate, you can get a formula for the observed information in terms of the MLE (which you have to calculate numerically):
$$\newcommand{\e}{\mathrm{e}}\newcommand{\d}{\operatorname{d}}f(x) = \frac{\e^{-\theta}\theta^x}{x!(1-e^{-\theta})}$$
$$\ell(\theta)=-\theta+ x\log\theta -\log(1-\e^{-\theta})$$
$$\frac{\d\ell(\theta)}{\d\theta} = -1 +\frac{x}{\theta} - \frac{\e^{-\theta}}{1-\e^{-\theta}}$$
$$I_1\left(\hat{\theta}\right)=-\frac{\d^2\ell\left(\hat\theta\right)}{\left(\d\hat\theta\right)^2} = \frac{x}{\hat\theta} - \frac{\e^{-\hat\theta}}{\left(1-\e^{-\hat{\theta}}\right)^2}$$
Notable cases excluded by the regularity conditions include those where


*

*the parameter $\theta$ determines the support of the data, e.g. sampling from a uniform distribution between nought and $\theta$

*the number of nuisance parameters increases with sample size

