Consider a linear model

$$y = X\beta + e$$

One can express the estimator for $\beta$ as

$$\hat{\beta} = A^Ty$$

It is stated in Hansen's Econometrics that

$$\mbox{Var} (\hat{\beta}|X) = \mbox{Var} (A^Ty|X) = \mbox{Var} (A^Te|X) = A^T D A$$

where $D$ is variance-covariance matrix with variances of error term on the diagonal and zeros everywhere else. So far I understand that the only step to prove it is simply to take out $A^T$ from $\mbox{Var} (A^Te|X)$ with squaring. But I don't quite understand why this step results into

$$\mbox{Var} (A^Te|X) = A^T \mbox{Var} (e|X)A$$

  • $\begingroup$ I'd break this into two steps/ideas. (1) Simple vector case. Let $\mathbf{w}$ be some vector. Convince yourself that $\mathrm{Var}\left( \mathbf{w}' \mathbf{e} \right) = \mathbf{w}' \mathrm{Var}\left( \mathbf{e} \right)\mathbf{w} $ (can even write it out as a sum $\sum_{i,j} w_i w_j \Sigma_{ij}$ where $\Sigma = \mathrm{Var}\left( \mathbf{e} \right)$. (2) Write matrix $A =\begin{bmatrix} \mathbf{w}_1 & \mathbf{w}_2 & \ldots & \mathbf{w}_n \end{bmatrix} $ in terms of its column vectors. The matrix case is just an extension of the first, vector case. $\endgroup$ – Matthew Gunn Oct 3 '16 at 16:42

Note that $$Var(A^Te\vert X) = \mathbb{E}[(A^Te)(A^Te)^T\vert X]$$

Assuming $\mathbb{E}(e)$ is a vector of zeros. So by expanding the above we see that:

$$Var(A^Te\vert X) = \mathbb{E}[(A^Te)(A^Te)^T\vert X] = \mathbb{E}[A^Tee^TA\vert X] = A^T\mathbb{E}[ee^T\vert X]A = A^TDA$$

The second to last equality above holds because $A = [X^TX]^{-1}X^T$ which means we can pull it out of the expectation conditioned on $X$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.