Multiple Linear Regression: Proof of Pythegore's Theorem

Question

How can I prove pythegore's theorem?

Where $y'$ is the transpose of $y $

And the $\hat{y}$ matrix is the matrix of the estimated values of $y$.

So far, I have:

$y'\hat{y} = y'y$

$ \implies (X\beta + \epsilon)' (X (X' X)^{-1} X' y) = (X\beta + \epsilon)' (y)$

$\implies \beta' X' X (X' X)^{-1} X' y + \epsilon ' X (X' X)^{-1} X' y = \beta' X' Y + \epsilon ' y$

$\implies \beta' X' Y + \epsilon ' X (X' X)^{-1} X' y = \beta' X' y + \epsilon' y$

but $\beta' X' Y$ is a $1 \times p$ times $p \times n$ times $n \times 1$ matrix i.e. a $1\times 1$ matrix i.e. a scalar, so I can simply subtract it from both sides, yielding

$\epsilon ' X (X' X)^{-1} X' y = \epsilon' y$

$\implies X (X' X)^{-1} X' =I$

i.e. the Hat matrix $H =I$ for ANY $n \times n$ identity matrix, which is obviously a nonsensical result.

Is this a better, more sound way of disproving this?

Answer 1

$x^TAy=x^Ty$ does not force $A=I$, e.g. $y$ can be an eigenvector of $A$ with $\lambda=1$. So, even if you show $H\neq I$, it doesn't prove anything. And, if you intend to go numerically, it's just one step away for you to really find that $y^T\hat{y}\neq y^Ty$.

However, it is true that $\hat{y}'\hat{y}=\hat{y}'y$ because the estimation method assumes that the error is orthogonal to the data (I think @Xian's comment is for this one), you can also check it by substitution easily. Maybe you were asking for it.

Note:

but 𝛽′𝑋′𝑌 is a 1×𝑝 times 𝑝×𝑛 times 𝑛×1 matrix i.e. a 1×1 matrix i.e. a scalar, so I can simply subtract it from both sides...

You can always do that. It doesn't have to be a scalar.