1
$\begingroup$

For the multiple linear regression, I have

$Y_{n\times 1} = X_{n\times k}\beta_{k\times 1} + \epsilon_{n\times 1}$ and $\hat{Y} = X\hat{B} $ where $\hat{B} = (X'X)^{-1}X'Y$

$\hat{Y} = X\hat{\beta} = X(X'X)^{-1}X'Y = HY$ where $H = X(X'X)^{-1}X' $ is symmetric, idempotent matrix

Similarly, $e = Y - \hat{Y} = Y - Hy = (I_n - H)Y = \overline{H} Y$ where $\overline{H} = I_n - H$ is again symmetric, idempotent matrix.

I am interested in calculating

$cov(\hat{Y}, e) = cov(HY, \overline{H}Y)$

How can I calculate this? Basically, how do I open this up so that I am able to use nice properties of $H$ and $\overline{H}$?

$\endgroup$

2 Answers 2

1
$\begingroup$

Say $X$ is a known matrix. We're looking for the cross-covariance matrix: $$\begin{align}cov(\hat Y,e)&=E[HYY^T\bar H^T]-E[HY]E[Y^T\bar H^T]\\&=H E[YY^T]\bar H-HE[Y]E[Y^T]\bar H\\&=H cov(Y)\bar H\end{align}$$

$\endgroup$
1
$\begingroup$

We can argue using properties of projections: $\hat Y=HY$ is a projection into a subspace $\mathbb{D}$ and $e=(I-H)Y$ is the projection into its orthogonal complement $\mathbb{D}^\perp$. Since any vector in $\mathbb{D}$ is orthogonal to any vector in $\mathbb{D}^\perp$, the covariance is identically zero.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.