I am trying to get a grasp on the ML estimator and the presentation of the asymptotic covariance matrix is really confusing to me. First, it is stated that the matrix is inverse of the information matrix. The information matrix is simply the variance covariance matrix of the partial derivatives of the log likelihood function (=score function). Although there appears to be a couple different ways of computing the information matrix (based on for example the 2nd derivative). First question, why are there different ways of calculating the same matrix?

Now, after this explanation I am naturally interested in deriving this matrix. Let's assume the normal distribution (ML being equivalent to OLS then). Then, the ML function is:

$$F(y_1... y_n|x_i; \beta, \sigma^2) = \prod_{i=1}^{n} \frac{1}{2\pi \sigma^2}e^{(-\large \frac{y_i - x_i \beta }{2 \sigma^2})^2}$$

Where $x_i$ and $y_i$ are the data points of the variables and beta and sigma the parameters to be estimated. Taking a log and then solving the partial derivatives per parameter and multiplying the resulting vector with it's transpose, taking the expected value and finally inverting the resulting matrix gives the asymptotic covariance matrix. According my calculation that is:

$$V = I(\beta, \sigma^2)^{-1} =\begin{bmatrix} \frac{\sigma^2}{x_i x_i'} & 0 \\ 0 & 2 \sigma^4 \end{bmatrix}$$

What I would hope to have is an explanation of how this matrix is exactly used. Is the first entry the asymptotic variance of the beta and the 2nd entry (2nd row/colmn) the variance of the variance (or the variance of the width of the normal distribution)? What would happen here if there were more B parameters? Just to be clear the sigma sign always refers to the width of the normal distribution? What does this matrix actually tell us (especially the variance of variance is a confusing concept)?

Finally, what is this matrix used for? For this sort of example you would rather use the covariance matrix of the OLS to estimate the variance of the beta parameters. And for the general case you would rather use the wald test or something similar for hypothesis testing. What is the purpose of deriving this matrix?

  • 2
    $\begingroup$ Please explain what $\beta, \sigma^2 x_i$ are in your example for the readers to be able to fully understand your question. $\endgroup$ – Greenparker Apr 18 '16 at 14:44
  • $\begingroup$ @Greenparker Hopefully it is clearer now. $\endgroup$ – Dole Apr 18 '16 at 14:58
  • 2
    $\begingroup$ If $\hat{\sigma}^2$ is the MLE for $\sigma^2$, then it is a statistic, and hence it is random. It changes from sample to sample, which means that it has a distribution, and asymptotically its variance is $2\sigma^4$. $\endgroup$ – Greenparker Apr 18 '16 at 15:20
  • 2
    $\begingroup$ (1) You'd typically use the chi-square distribution to calculate confidence intervals for an unbiased estimate of the normal variance parameter $\sigma^2$, rather than use the asymptotic results for the MLE. (2) Uncertainty about the true value of $\sigma^2$ certainly does affect the precision of estimates of the regression coefficients - hence the use of Student's t-distribution in the calculation of confidence intervals. $\endgroup$ – Scortchi Apr 18 '16 at 16:29
  • 1
    $\begingroup$ Asymptotic results are only approximate in finite samples. When exact results are easily got, people prefer to use them. $\endgroup$ – Scortchi Apr 19 '16 at 10:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.