# Why is $X\hat{\beta}$ regarded as $y$ in multiple linear regression while estimating sigma square?

Screenshot from page 80 of the textbook "Introduction to Linear Regression Analysis" fifth edition by Douglas C. Montgomery

Let $$X$$ be $$n \times p$$, $$y$$ and $$\hat{y}$$ be $$n \times 1,$$ and $$\hat{\beta}$$ be $$p \times 1$$ matrix in the multiple linear regression model. From the matrix calculation, we can easily find $$\hat{\beta} = (X'X)^{-1}X'y$$ and $$\hat{y}=X\hat{\beta}$$. However, in the later part of this chapter, the estimation of $$\sigma^{2}$$, while calculating the residual sum of squares, the textbook says $$X'X\hat{\beta} = X'y$$ which indicates $$X\hat{\beta}=y$$. But I don't understand why it is not $$\hat{y}$$. Can anyone answer this question?

$$X'X\hat{\beta} = X'X \left((X'X)^{-1}X'y\right)=\left(X'X (X'X)^{-1}\right)X'y=X'y$$

This does not imply that $$X\hat{\beta}=y$$ though. In algebra, the statement $$CA=CB$$ only implies that $$A=B$$ if $$C$$ is invertible, and, in this case, $$C=X'$$ is not even (necessarily) a square matrix.

What it does imply, however, is that $$X'y = X'\hat y$$, which is true in general, since $$X'e = X'y-X'X\hat \beta=0$$.

If $$X$$ is a square matrix with full-rank ($$n=p$$), then $$y=\hat y$$.
• Thank you for answering this question clearly. By the way, is it the typo in the first line that you wrote $X'y$ outside the bracket from $X'X((X'X)^{-1}X'y)X'y$? Would it be $X'X((X'X)^{-1}X'y)$? Commented Apr 13, 2023 at 7:02
• For a specific example, consider the case of a regression of $y$ on a constant, $X=\mathbf{1}$. Then $\hat\beta=\bar y$ and thus $X\hat\beta=\mathbf{1}\bar{y}$, the sample mean replicated into a vector of ones, which would only be equal to $y$ if $y$ always took the same value. Commented Apr 13, 2023 at 12:54