# What does the asymptotic heteroskedastic variance-covariance matrix of beta look like?

I don't really understand the variance-covariance matrix of beta when there is heteroscedasticity. Can someone explain what the matrix looks like? I am talking about:
$$V_\beta=Q_{xx}^{-1}\Omega Q_{xx}^{-1}$$

where,

• $V_\beta$ is the variance of beta (asymptotic, so equal to the population variance for large n).
• $Q_{xx}$ is $X'X$ and $\Omega$ is the variance of $X'e$ where the estimator for $\beta$ is $(X'X)^{-1}X'y$ and $e=y-X\beta$. Therefore $y=e+X\beta$ is substituted for $Y$ and then solved for $\hat\beta-\beta$, getting $\hat\beta-\beta=(X'X)^{-1}X'e$.

So if I understand correctly, the variance-covariance matrix shows the variance on the diagonal of $\beta$, but what are on the off diagonal elements?

• Can you expand on what the terms in your formula stand for? Telling us how far you have got so far in your thinking would also be useful Commented Oct 16, 2016 at 11:08
• Hint: $\boldsymbol{\beta}$ is a vector. Commented Oct 16, 2016 at 17:04

Ok, so start with $$\hat\beta = (X^TX)^{-1}X^TY$$ and write this as $$\hat\beta=AY$$.

We now have $$\mathrm{var}[\hat\beta] = A^T\mathrm{var}[Y]A$$ as a property of linear transformations, so $$\mathrm{var}[\hat\beta] = (X^TX)^{-1} X^T\mathrm{var}[Y]X (X^TX)^{-1}$$ You have this re-grouped as $$\mathrm{var}[\hat\beta] = (X^TX)^{-1} \left(X^T\mathrm{var}[Y]X\right) (X^TX)^{-1}$$

So, what does the matrix $$\mathrm{var}[Y]$$ look like? If different $$Y$$s are independent, the matrix is diagonal, with the variances of individual $$Y$$s as the diagonal elements. $$\Omega$$ doesn't have any very illuminating form; it's a sum of matrices of the form $$x_ix_i^T$$ weighted by the individual variances of the observations.

If the variances are all the same, $$\mathrm{var}[Y]$$ is just $$\sigma^2$$ times an $$n\times n$$ identity matrix and the formula simplifies to the familiar homoscedastic formula. $$\Omega$$ is then just $$\sigma^2 X^TX$$.

Asymptotically, it's convenient to put in some $$n$$s and consider $$n\mathrm{var}[\hat\beta]= \left(\frac{1}{n} X^TX)\right)^{-1} \left(\frac{1}{n}X^T\mathrm{var}[Y]X\right)\left(\frac{1}{n} X^TX\right)^{-1}$$ This has the advantage that all the bits are means and actually converge to non-zero constants (under some assumptions) with $$n\mathrm{var}[\hat\beta]$$ being the limiting variance of $$\sqrt{n}(\hat\beta-\beta_0)$$. By contrast $$\mathrm{var}[\hat\beta]$$ just converges to zero; $$\mathrm{\hat\beta}$$ gets less and less variable as $$n$$ increases.

Finally, you might also be interested in the heteroscedasticity-consistent estimator that's a plug-in version of this. The estimator has the same $$X^TX$$ matrix $$\widehat{\mathrm{var}}[\hat\beta] = (X^TX)^{-1} \left(X^T\widehat{\mathrm{var}}[Y]X\right) (X^TX)^{-1}$$ Here, $$\widehat{\mathrm{var}}[Y]$$ is again a diagonal matrix, with the diagonal elements being $$(y_i-x_i\hat\beta)^2$$. Again, if you want to work asymptotically it's convenient to put in $$1/n$$s everywhere so that the components of the estimator are each means that have straightforward limits.

As tchakravarty noticed, $\beta$ is a column vector of all the parameters of the model. So the variance-covariance matrix shows on the diagonals the covariance of the $\beta$ with itself (=variance), and on the off diagonal elements it shows the covariance of individual $\beta$'s with other $\beta$'s.

In conclusion shows this matrix the asymptotic heteroskedastic variance of the estimator $\hat \beta$, which converges by the Central Limit Theorem (CLT) and the Weak Law of Large Numbers (WLLN) to the variance of the population $\beta$.

• In what sense are the parameters--which are considered fixed--supposed to be a "population"??
– whuber
Commented Oct 16, 2016 at 21:08