In a linear regression, suppose we have $N=500$ observations. Then suppose that I ran a regression of dependent $Y$ against independent $X_1, \ldots, X_p$, where $p=500$ as well. What happens in this case and how would it relate to a residual regression formula? I'm also wondering what's the effect on R^2?

  • 2
    $\begingroup$ Does this answer your question? What is the R squared of a regression where none of the variables are collinear?: if by "independent", you mean they are linearly independent as vectors in $\mathbb{R}^{500}$. $\endgroup$
    – Zhanxiong
    Feb 7, 2023 at 22:30
  • $\begingroup$ @Zhanxiong Yes, how can I also relate it to the residual regression formula? $\endgroup$
    – user321627
    Feb 7, 2023 at 22:44
  • $\begingroup$ The residual will be $0$. An on-line answer is: the projection (hat) matrix $H = X(X^TX)^{-1}X^T = XX^{-1}(X^T)^{-1}X^T = I_{(500)}$ under your assumption (because $X$ is invertible), whence the residual vector $\hat{\epsilon} = y - Hy = y - y = 0$, which further implies $R^2 = 1$. $\endgroup$
    – Zhanxiong
    Feb 7, 2023 at 22:47
  • $\begingroup$ @Zhanxiong May I ask which part of the above is the residual regression formula? $\endgroup$
    – user321627
    Feb 8, 2023 at 0:40
  • 1
    $\begingroup$ "$\hat{\epsilon} = y - Hy$" is the formula to calculate residual (this is in matrix form). You may want to read this section if this sounds unfamiliar. $\endgroup$
    – Zhanxiong
    Feb 8, 2023 at 1:04


Browse other questions tagged or ask your own question.