Two important properties of the maximum likelihood estimator (MLE) are functional invariance and asymptotic normality.

Functional invariance: If $\hat{\theta}$ is the MLE for $\theta$, and if $g(\theta)$ is any transformation of $\theta$, then the MLE for $\alpha = g(\theta)$ is $\hat{\alpha} = g(\hat{\theta})$.

Asymptotic normality: If $\hat{\theta}$ is the MLE for $\theta$ then $\sqrt{n}(\hat{\theta} - \theta) \xrightarrow[]{d} \mathcal{N}(0, I^{-1})$, where $I$ is the Fisher information matrix.

My question is what happens when $\alpha = 1/\theta$?

If $\hat{\theta}$ is the MLE for $\theta$, then by the functional invariance property $\hat{\alpha} = 1/\hat{\theta}$. The asymptotic normality property means that

  1. $\hat{\theta}$ is asymptotically normally distributed about $\theta$;
  2. $\hat{\alpha} = 1/\hat{\theta}$ is asymptotically normally distributed about $1/\theta$.

To me this looks to be a contradiction since the reciprocal of a normal random variable is not itself normally distributed.


It's possible situation because of asymptotics.

When the $\hat \theta$ is close enough to $\theta$, $\hat \alpha= \frac{1}{\hat \theta}= \frac{1}{\theta} \left(1+\frac{\hat \theta -\theta}{\theta} \right)^{-1} \simeq \frac{1}{\theta} \left(1-\frac{\hat \theta -\theta}{\theta} \right)= \frac{1}{\theta}-\frac{1}{\theta^2}(\hat \theta -\theta)=\alpha-\frac{1}{\theta^2}(\hat \theta -\theta)$

$\hat \alpha -\alpha\simeq -\frac{1}{\theta^2}(\hat \theta -\theta)$

I guess this equation doesn't look to be a contradiction.

And actually $-\frac{1}{\theta^2}$ is $g'(\theta)$. This result can be generally applied.

I am not a native English speaker and also a self-study learner. I hope this answer is helpful.

| cite | improve this answer | |
  • $\begingroup$ Is the general result the Delta Method? $\endgroup$ – Estacionario Aug 28 '18 at 14:56
  • 1
    $\begingroup$ Yes. The same principle is used. $\endgroup$ – KDG Aug 28 '18 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.