I have a single parameter $\theta > 0$ of a probability model I estimate with MLE on i.i.d. data. To get rid of the positivity constraint I instead estimate $\log \theta$ for which MLE gives me an estimate $\log \hat{\theta}_n$. We know (assuming the appropriate conditions) $$ \sqrt{n}(\log \hat{\theta}_n - \log \theta) \xrightarrow d \mathcal{N}\left(0, \frac{1}{F}\right) $$ where $F$ is the Fisher information. As I understand it, for large $n$ we may then estimate the precision of the estimate $\log \hat{\theta}_n$ by assuming it is approximately normal, with mean $\log \theta$ and variance $\frac{1}{nF}$.

Here's where I'm confused. On one hand, assuming $\log \hat{\theta}_n$ is normal with mean $\log \theta$ and variance $\frac{1}{nF}$, then $\hat{\theta}_n$ is lognormal with parameters $\log \theta$ and $\frac{1}{nF}$. On the other hand, by the Delta method we know $$ \sqrt{n}(\hat{\theta}_n - \theta) \xrightarrow d \mathcal{N}\left(0, \frac{\theta^2}{F}\right), $$ and so for large $n$ we can alternatively estimate the precision of $\hat{\theta}_n$ as approximately normal with mean $\theta$ and variance $\frac{\theta^2}{nF}$.

So it seems we have two potential (approximate) distributions for $\hat{\theta_n}$: \begin{align*} \hat{\theta_n} & \sim \log \mathcal{N}\left(\log \theta, \frac{1}{nF}\right), \\ \hat{\theta_n} & \sim \mathcal{N} \left(\theta, \frac{\theta^2}{nF}\right). \end{align*}

I feel uneasy about this because it seems the lognormal is more natural in this case because $\theta > 0$, but then again the Delta method seems correct, too.

  • 3
    $\begingroup$ Asymptotically these distributions must be the same--and asymptotic results are the only things assured by ML theory in the first place. But please be careful: your symbol "$F$" refers to different values depending on whether $\theta$ or $\log(\theta)$ is used as the parameter. $\endgroup$ – whuber Jan 29 '16 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.