I have a question to solve the following matrix problem:

$$E[( (X'X)^{-1}X \epsilon )^T ((X'X)^{-1}X \epsilon )]$$ into the solution $$= \Sigma^{2} (X'X)^{-1}.$$

Where $\Sigma$ is the covariance matrix of $\epsilon$.

My question is how to solve this problem using the properties of matrix algebra.

This equation occurs when using least squares to solve a linear regression problem. (as can be seen in: http://en.wikipedia.org/wiki/Proofs_involving_ordinary_least_squares , section Unbiasedness and Variance of $\hat\beta$).

I saw a solution for a specific case in this link: Expected Value and Variance of Estimation of Slope Parameter β1 in Simple Linear Regression. However, I was interested in the generalized solution described on the equation.

That is what I obtained so far:

Using properties of transpose, we obtain:

$$E[( (X'X)^{-1}X \epsilon )^T ((X'X)^{-1}X \epsilon )] = E[( (\epsilon' X' ((X'X)^{-1})' )((X'X)^{-1}X \epsilon )].$$

$(X'X)^{-1}$ is a symmetric matrix, thus: $$( (X'X)^{-1} )' = (X'X)^{-1}.$$

And I obtain $$E[( (X'X)^{-1}X \epsilon )^T ((X'X)^{-1}X \epsilon )] = E[( (\epsilon' X' (X'X)^{-1})^{2} X \epsilon )].$$

I could not solve any further than this. I also tried the fact a symmetric matrix, $S$, can be transformed in $S = QDQ'$, for some orthogonal matrix $Q$ and diagonal matrix $D$.

Can anyone help me with this solution?

  • $\begingroup$ You've copied the initial expectation incorrectly from the other page. This will change some of your work lower in your question. $\endgroup$ – Max Jan 9 '13 at 22:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.