Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I'm trying to see why this is the case. I have taken the LHS and added $X\hat{\beta}$ and subtracted $X\hat{\beta}$.
From this I can get both terms on the right but I end up with a cross product term
$2(y-X\hat{\beta})^T(X\hat{\beta} - X\beta)$
that I can't prove equals to zero. I think there is some trick I'm missing, anybody remember it?
$$2(\mathbf y-\mathbf X\hat{\beta})^T(\mathbf X\hat{\beta} - \mathbf X\beta)= 2(\mathbf y-\mathbf {\hat y})^T(\mathbf X\hat{\beta} - \mathbf X\beta)$$
$$=2\mathbf {\hat \varepsilon}^T \mathbf X(\hat{\beta} - \beta) = 2(\mathbf X^T \mathbf {\hat \varepsilon})^T (\hat{\beta} - \beta)=2\cdot \mathbf 0\cdot(\hat{\beta} - \beta) =0 $$
since the residuals are by construction orthogonal to the regressor matrix.
