3
$\begingroup$

From Wikipedia

Under the Wald statistical test, the maximum likelihood estimate $\hat\theta$ of the parameter(s) of interest $\theta$ is compared with the proposed value $\theta_0$, ... In the univariate case, the Wald statistic is $$ \frac{ ( \widehat{ \theta}-\theta_0 )^2 }{\operatorname{var}(\hat \theta )} $$ ... where $\operatorname{var}(\widehat\theta)$ is the variance of the maximum likelihood estimate.

My question is about how to estimate $\operatorname{var}(\widehat\theta)$, i.e. the variance of the MLE $\hat{\theta}$?

  1. The same article says

    A reasonable estimate of the variance for the MLE can be given by $\frac{1}{I_n(\widehat\theta)}$ , where $I_n$ is the Fisher information of the parameter.

  2. I am thinking of another way. Since $\operatorname{var}(\widehat\theta)$ is $f(n, \theta, \eta)$, i.e. some function of the sample size $n$ and the true value $\theta$ of the parameter of interest and some other parameter $\eta$ (because $\theta$ alone may not be fully able to specify the distribution of the sample), can we first estimate the MLEs of $\theta$ and $\eta$ as $\hat{\theta}$ and $\hat{\eta}$, and then will $f(n, \hat{\theta}, \hat{\eta})$ be the MLE of $\operatorname{var}(\widehat\theta)$ because of the equivariance of MLE?
  3. Will the above two methods be the same (in some cases, such as when the statistical model of the sample is normal distributions?)

Thanks and regards!

$\endgroup$
3
$\begingroup$

The method you mention in 2 is similar to the way the variance is usually estimated--through the observed information. The problem with the Fisher Information is that it depends upon an unknown parameter, $\theta$. Using the plug-in estimator defined in the link I specified will allow you to compute an estimate of the Fisher Information. The second Wikipedia quote you mention is equivalent to (2.).

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks! The observed information depends on both the sample and the true unknown parameter value. To estimate the Fisher information via the observed information, shall we plug in the MLE of the unknown parameter value? $\endgroup$ – Tim Jun 1 '13 at 17:18
  • 2
    $\begingroup$ First you would need to write down the likelihood, then find the matrix of second derivatives with respect to the true parameter. This should give you a matrix with the elements (most likely) relying on the true unknown parameter. It's that matrix that you would plug the MLE to get an estimate of the Fisher Information. $\endgroup$ – Christopher Aden Jun 1 '13 at 17:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.