0
$\begingroup$

Upon studying the ML estimator this concept still confuses me.

First define an asymptotic covariance matrix for the MLE estimator (just as an example, we have two parameters $\beta$ and $\sigma^2$, $x_i$ is a vector of observations, see the derivation at the bottom):

$$V = I(\beta, \sigma^2)^{-1} =\begin{bmatrix} \frac{\sigma^2}{x_i x_i'} & 0 \\ 0 & 2 \sigma^4 \end{bmatrix}$$

As it can be seen the variables are uncorrelated, given that the variance is per definition obtained from the matrix, and our assumption is that it's normally distributed:

$$ \sqrt N (\beta_p-\hat \beta) \rightarrow ^d N(0,\frac{\sigma^2}{x_i x_i'})$$

$$ \sqrt N (\sigma_p^2 - \hat \sigma^2) \rightarrow ^d N(0, 2 \sigma^4)$$

My confusion is that generally speaking, shouldn't asymptotic variance matrix be defined as something that happens as N grows. Thus the diagonals of the matrix should be divided by N (and perhaps even probability limit taken to obtain the true asymptotic value, which in this case would be 0 for both estimates).

Are the different definitions at odds here or am I misunderstanding something here?

$\\$

Here is the full derivation of obtaining the information matrix, since it could be a other source of confusion:

$$L(y_1... y_n|x_i; \beta, \sigma^2) = \prod_{i=1}^{n} \frac{1}{2\pi \sigma^2}e^{(-\large \frac{y_i - x_i' \beta }{2 \sigma^2})^2}$$

Take logs and then the derivative, to get the score vector:

$$ \frac{\delta (log L)}{\delta \hat \beta} = \sum_{i=1}^nx_i(\frac {y_i-x_i' \hat\beta}{\sigma^2})$$

$$ \frac{\delta (log L)}{\delta \hat \beta} = \frac {N}{2 \sigma^2} +\sum_{i=1}^nx_i(\frac {y_i-x_i' \hat\beta}{2 \sigma^4})^2$$

At this point it's somehow fine to remove the sum signs, don't know what that is based on (please inform me if you do). Finally, put the two results into a vector and multiply the vector with its inverse, then take the expectation:

$$\large \text E [\begin{bmatrix} x_i(\frac {y_i-x_i' \hat\beta}{\sigma^2}) \\ \frac {N}{2 \sigma^2} +(\frac {y_i-x_i' \hat\beta}{2 \sigma^4})^2 \end{bmatrix}\begin{bmatrix} x_i'(\frac {y_i-x_i' \hat\beta}{\sigma^2}) & \frac {N}{2 \sigma^2} +(\frac {y_i-x_i' \hat\beta}{2 \sigma^4})^2 \end{bmatrix}]$$

Calculating the result will result in the information matrix, the asymptotic covariance matrix being its inverse.

$\endgroup$
  • $\begingroup$ You're missing a lot of context and definitions that are needed to make sense of this question. What does "$x_i x_i^\prime$" mean? What is "$\beta$"? $\endgroup$ – whuber Apr 18 '16 at 22:20
  • $\begingroup$ Thanks for the edits. I am still a little taken aback by the presence of the superfluous subscript "$i$". And for $V$ to be the information matrix I believe you must be assuming the observations are univariate, independent, and Normally distributed: not all information matrices for two-parameter Maximum Likelihood problems look like this! Presumably "$N$" (on the left hand sides, anyway) is the count of those observations. Now, as to your question: what exactly do you think those arrows mean if they do not refer to arbitrarily large values of $N$?? $\endgroup$ – whuber Apr 18 '16 at 22:38
  • $\begingroup$ @whuber The information matrix is derived using the BHHH estimator, using the normal distribution likelihood function (equivalent to OLS). The subscript merely carries over from $x_i' \beta$, which is the regression equation. I think the arrows are referring to the shape of the distribution, not it's actual variance (which must change as N grows - that result can't be right). $\endgroup$ – Dole Apr 18 '16 at 22:55
  • 1
    $\begingroup$ It sounds like you may have some confusion over the notion of convergence in distribution. Typically in problems of interest, if you have a quantity whose mean is fixed but whose variance decreases as $\frac{1}{n}$, it will converge to a constant; this is not useful. By subtracting that mean and multiplying the difference by $\sqrt{n}$, you can get convergence to a fixed distribution. $\endgroup$ – Glen_b Apr 19 '16 at 12:55
  • 1
    $\begingroup$ Consider a simple mean, $\mu$ for iid observations with mean $\mu$ & variance $σ^2$. As $n\to\infty$, $\bar{X}_n$ converges to a point mass at $\mu$ which is not especially useful for use in calculations at large $n$, while $\sqrt{n}(\bar{X}_n-\mu)$ gives convergence to $N(0,σ^2)$. If you're interested in the variance of $\bar{X}$ at some finite $n$, then yes, you'd divide $σ^2$ back by $n$ again. There's nothing actually confusing in what's happening aside from such inconsistencies as the way people use terms like asymptotic variance, but usually it's either defined or obvious from context. $\endgroup$ – Glen_b Apr 19 '16 at 23:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.