# Definition of asymptotic variance

Upon studying the ML estimator this concept still confuses me.

First define an asymptotic covariance matrix for the MLE estimator (just as an example, we have two parameters $\beta$ and $\sigma^2$, $x_i$ is a vector of observations, see the derivation at the bottom):

$$V = I(\beta, \sigma^2)^{-1} =\begin{bmatrix} \frac{\sigma^2}{x_i x_i'} & 0 \\ 0 & 2 \sigma^4 \end{bmatrix}$$

As it can be seen the variables are uncorrelated, given that the variance is per definition obtained from the matrix, and our assumption is that it's normally distributed:

$$\sqrt N (\beta_p-\hat \beta) \rightarrow ^d N(0,\frac{\sigma^2}{x_i x_i'})$$

$$\sqrt N (\sigma_p^2 - \hat \sigma^2) \rightarrow ^d N(0, 2 \sigma^4)$$

My confusion is that generally speaking, shouldn't asymptotic variance matrix be defined as something that happens as N grows. Thus the diagonals of the matrix should be divided by N (and perhaps even probability limit taken to obtain the true asymptotic value, which in this case would be 0 for both estimates).

Are the different definitions at odds here or am I misunderstanding something here?

$\\$

Here is the full derivation of obtaining the information matrix, since it could be a other source of confusion:

$$L(y_1... y_n|x_i; \beta, \sigma^2) = \prod_{i=1}^{n} \frac{1}{2\pi \sigma^2}e^{(-\large \frac{y_i - x_i' \beta }{2 \sigma^2})^2}$$

Take logs and then the derivative, to get the score vector:

$$\frac{\delta (log L)}{\delta \hat \beta} = \sum_{i=1}^nx_i(\frac {y_i-x_i' \hat\beta}{\sigma^2})$$

$$\frac{\delta (log L)}{\delta \hat \beta} = \frac {N}{2 \sigma^2} +\sum_{i=1}^nx_i(\frac {y_i-x_i' \hat\beta}{2 \sigma^4})^2$$

At this point it's somehow fine to remove the sum signs, don't know what that is based on (please inform me if you do). Finally, put the two results into a vector and multiply the vector with its inverse, then take the expectation:

$$\large \text E [\begin{bmatrix} x_i(\frac {y_i-x_i' \hat\beta}{\sigma^2}) \\ \frac {N}{2 \sigma^2} +(\frac {y_i-x_i' \hat\beta}{2 \sigma^4})^2 \end{bmatrix}\begin{bmatrix} x_i'(\frac {y_i-x_i' \hat\beta}{\sigma^2}) & \frac {N}{2 \sigma^2} +(\frac {y_i-x_i' \hat\beta}{2 \sigma^4})^2 \end{bmatrix}]$$

Calculating the result will result in the information matrix, the asymptotic covariance matrix being its inverse.

• You're missing a lot of context and definitions that are needed to make sense of this question. What does "$x_i x_i^\prime$" mean? What is "$\beta$"? – whuber Apr 18 '16 at 22:20
• Thanks for the edits. I am still a little taken aback by the presence of the superfluous subscript "$i$". And for $V$ to be the information matrix I believe you must be assuming the observations are univariate, independent, and Normally distributed: not all information matrices for two-parameter Maximum Likelihood problems look like this! Presumably "$N$" (on the left hand sides, anyway) is the count of those observations. Now, as to your question: what exactly do you think those arrows mean if they do not refer to arbitrarily large values of $N$?? – whuber Apr 18 '16 at 22:38
• @whuber The information matrix is derived using the BHHH estimator, using the normal distribution likelihood function (equivalent to OLS). The subscript merely carries over from $x_i' \beta$, which is the regression equation. I think the arrows are referring to the shape of the distribution, not it's actual variance (which must change as N grows - that result can't be right). – Dole Apr 18 '16 at 22:55
• It sounds like you may have some confusion over the notion of convergence in distribution. Typically in problems of interest, if you have a quantity whose mean is fixed but whose variance decreases as $\frac{1}{n}$, it will converge to a constant; this is not useful. By subtracting that mean and multiplying the difference by $\sqrt{n}$, you can get convergence to a fixed distribution. – Glen_b Apr 19 '16 at 12:55
• Consider a simple mean, $\mu$ for iid observations with mean $\mu$ & variance $σ^2$. As $n\to\infty$, $\bar{X}_n$ converges to a point mass at $\mu$ which is not especially useful for use in calculations at large $n$, while $\sqrt{n}(\bar{X}_n-\mu)$ gives convergence to $N(0,σ^2)$. If you're interested in the variance of $\bar{X}$ at some finite $n$, then yes, you'd divide $σ^2$ back by $n$ again. There's nothing actually confusing in what's happening aside from such inconsistencies as the way people use terms like asymptotic variance, but usually it's either defined or obvious from context. – Glen_b Apr 19 '16 at 23:44