What does the HAT matrix looks like when the residual sum of square equals zero? How do the predicted y and the measured y relate to each other in this case?


$H = X (X^T X)^{-1} X^T$. $Y$ doesn't enter into this formula, so neither does the sum of squares. If the sum of squares is zero because there are too many parameters, then $H$ won't be well-defined, but this is due to $X$ being rank-deficient and not because the sum of squares is zero. For the second question, if $RSS = 0$ then $\hat{y}_i = y_i$ for every $i$, correct?

  • $\begingroup$ For the residual sum of squares to be zero, it must be the case that each single residual must be zero. This will happen if and only if $X$ has number of columns equal to the number of observations. If it also has full column rank, the system will have an exact unique solution, and so all residuals will be zero. The Projection matrix will be perfectly well defined. It is the number of columns that creates the phenomenon of zero residuals. $\endgroup$ – Alecos Papadopoulos Jul 19 '15 at 22:44
  • 1
    $\begingroup$ No, it just means all the observations fall exactly on the regression line, which can happen even if $n > p$. $\endgroup$ – dsaxton Jul 19 '15 at 22:51
  • $\begingroup$ Certainly, but a constant sample, comprised of totally identical observations is a rather useless and uninteresting situation. This is why I never bother to declare "assuming a non-identical sample..." $\endgroup$ – Alecos Papadopoulos Jul 19 '15 at 22:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.