2
$\begingroup$

Two important properties of the maximum likelihood estimator (MLE) are functional invariance and asymptotic normality.

Functional invariance: If $\hat{\theta}$ is the MLE for $\theta$, and if $g(\theta)$ is any transformation of $\theta$, then the MLE for $\alpha = g(\theta)$ is $\hat{\alpha} = g(\hat{\theta})$.

Asymptotic normality: If $\hat{\theta}$ is the MLE for $\theta$ then $\sqrt{n}(\hat{\theta} - \theta) \xrightarrow[]{d} \mathcal{N}(0, I^{-1})$, where $I$ is the Fisher information matrix.

My question is what happens when $\alpha = 1/\theta$?

If $\hat{\theta}$ is the MLE for $\theta$, then by the functional invariance property $\hat{\alpha} = 1/\hat{\theta}$. The asymptotic normality property means that

  1. $\hat{\theta}$ is asymptotically normally distributed about $\theta$;
  2. $\hat{\alpha} = 1/\hat{\theta}$ is asymptotically normally distributed about $1/\theta$.

To me this looks to be a contradiction since the reciprocal of a normal random variable is not itself normally distributed.

$\endgroup$

1 Answer 1

1
$\begingroup$

It's possible situation because of asymptotics.

When the $\hat \theta$ is close enough to $\theta$, $\hat \alpha= \frac{1}{\hat \theta}= \frac{1}{\theta} \left(1+\frac{\hat \theta -\theta}{\theta} \right)^{-1} \simeq \frac{1}{\theta} \left(1-\frac{\hat \theta -\theta}{\theta} \right)= \frac{1}{\theta}-\frac{1}{\theta^2}(\hat \theta -\theta)=\alpha-\frac{1}{\theta^2}(\hat \theta -\theta)$

$\hat \alpha -\alpha\simeq -\frac{1}{\theta^2}(\hat \theta -\theta)$

I guess this equation doesn't look to be a contradiction.

And actually $-\frac{1}{\theta^2}$ is $g'(\theta)$. This result can be generally applied.

I am not a native English speaker and also a self-study learner. I hope this answer is helpful.

$\endgroup$
2
  • $\begingroup$ Is the general result the Delta Method? $\endgroup$ Aug 28, 2018 at 14:56
  • 1
    $\begingroup$ Yes. The same principle is used. $\endgroup$
    – KDG
    Aug 28, 2018 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.