# Multiple Linear Regression: Proof of Pythegore's Theorem

How can I prove pythegore's theorem?

Where $$y'$$ is the transpose of $$y$$

And the $$\hat{y}$$ matrix is the matrix of the estimated values of $$y$$.

So far, I have:

$$y'\hat{y} = y'y$$

$$\implies (X\beta + \epsilon)' (X (X' X)^{-1} X' y) = (X\beta + \epsilon)' (y)$$

$$\implies \beta' X' X (X' X)^{-1} X' y + \epsilon ' X (X' X)^{-1} X' y = \beta' X' Y + \epsilon ' y$$

$$\implies \beta' X' Y + \epsilon ' X (X' X)^{-1} X' y = \beta' X' y + \epsilon' y$$

but $$\beta' X' Y$$ is a $$1 \times p$$ times $$p \times n$$ times $$n \times 1$$ matrix i.e. a $$1\times 1$$ matrix i.e. a scalar, so I can simply subtract it from both sides, yielding

$$\epsilon ' X (X' X)^{-1} X' y = \epsilon' y$$

$$\implies X (X' X)^{-1} X' =I$$

i.e. the Hat matrix $$H =I$$ for ANY $$n \times n$$ identity matrix, which is obviously a nonsensical result.

Is this a better, more sound way of disproving this?

• Add the self study tag. Commented Oct 4, 2019 at 1:19
• I've tidied up the mathematical text. Please do make sure it gives your same calculations. You may be able to see the pattern of what I did if you have to edit a typo of mine, but if you don't point me to the correction for me to make it. Write "@Dave" at the beginning of your response so I get an alert that you've replied. I do want to remark about point #2. If you can find $X$ such that $H \ne I$, then you're done (assuming the rest of the calculation is true), yes. However, you don't get to adjust $n$ willy nilly. $n$ is fixed by the number of columns in $X$.
– Dave
Commented Oct 4, 2019 at 1:42
• This is called Pythagore's theorem. Commented Oct 4, 2019 at 5:12

$$x^TAy=x^Ty$$ does not force $$A=I$$, e.g. $$y$$ can be an eigenvector of $$A$$ with $$\lambda=1$$. So, even if you show $$H\neq I$$, it doesn't prove anything. And, if you intend to go numerically, it's just one step away for you to really find that $$y^T\hat{y}\neq y^Ty$$.
However, it is true that $$\hat{y}'\hat{y}=\hat{y}'y$$ because the estimation method assumes that the error is orthogonal to the data (I think @Xian's comment is for this one), you can also check it by substitution easily. Maybe you were asking for it.