Identity in Simple Linear Model I'm working on this identity 
$$\sum_{i-1}^n (y_i - \hat {\beta_0} - \hat {\beta_1}x_i)^2 = \sum_{i=1}^n y_i^2 - \hat {\beta_0}\sum_{i=1}^n y_i - \hat {\beta_1} \sum_{i=1}^n x_iy_i$$
I have these relationships to work with:
$$\hat {\beta_1} = \frac {    n\sum_{i_1}^n x_iy_i - \left ( \sum_{i=1}^n x_i \right ) \left ( \sum_{i=1}^n y_i \right )}{  n \left ( \sum_{i=1}^n x_i^2 \right ) -\left  (\sum_{i=1}^n x_i \right )^2 }$$
$$ \hat {\beta_0}= \overline {y} - \hat {\beta_1} \overline {x}$$
A little manipulation also shows 
$$\hat {\beta_1}= \frac {         \sum_{i=1}^n  ( x_i - \overline {x}) y_i}{  \left ( \sum_{i=1}^n x_i^2  \right ) -n \overline {x}^2   }$$
My strategy has to substitute $\hat {\beta_1}$ out of the equation, but I just keep getting $\overline {y}$ in too many terms and not any terms with $y_i$. The formula for $\hat {\beta_1}$ is too complicated to consider working with. What am I missing?
 A: You are trying to reprove the Pythagorean Theorem.  Understanding the connection provides a powerful intuition for understanding ordinary least squares regression and (incidentally) makes short work of the proof.

Let $y$ represent the vector $(y_1, y_2, \ldots, y_n)$, $\mathbf{1}$ the $n$-vector $(1,1,\ldots,1)$, and $x$ the vector $(x_1,x_2, \ldots, x_n)$.  Denote by $\hat{y}$ the vector $\hat\beta_0\mathbf{1} + \hat\beta_1 x.$  In this notation the identity (after combining the last two sums) is
$$ ||y-\hat y||^2 = \sum_{i-1}^n (y_i - \hat {\beta_0} - \hat {\beta_1}x_i)^2 = \sum_{i=1}^n y_i^2 - \sum_{i=1}^n\left(\hat {\beta_0} - \hat {\beta_1} x_i\right)y_i = ||y||^2 - \hat y \cdot  y.$$
Because $\hat\beta_0$ and $\hat\beta_1$ are formulas for the least-squares coefficients, by definition $y-\hat{y}$ minimizes the squared distance from $y$ to the line generated by $\hat{y}$.  Because $y$ and $\hat{y}$ span a space of at most two dimensions, understanding their relationship is a matter of planar Euclidean geometry which is faithfully illustrated with a simple diagram:

The formulae for $\hat\beta_0$ and $\hat\beta_1$ are usually derived by demonstrating the geometrically obvious fact that $y-\hat y$ must be perpendicular to $\hat y$.  In terms of vector operations, this means their dot product is zero:
$$\hat y \cdot (y - \hat y) = 0.$$
Expanding this dot product shows it is equivalent to the key relationship
$$||\hat y||^2 = \hat y \cdot \hat y = \hat y \cdot y.$$
Consider the Pythagorean Theorem.  In this right triangle it asserts that the square of one leg equals the square of the hypotenuse minus the square of the other leg:
$$||y-\hat y||^2 = ||y||^2 - ||\hat y||^2.$$
The key relationship provides another expression for the last term, yielding
$$||y-\hat y||^2 = ||y||^2 - \hat y \cdot  y,$$
which is the desired identity, QED.
A: OK, I will do some parts and leave the rest for you to do it yourself. I dropped the index of summations for simplicity. Start from expanding the L.H.S to have $$L.H.S=\sum y_i^2+\sum(\hat{\beta_0}+\hat{\beta_1}x_i)^2-2\hat{\beta_0}\sum y_i-2\hat{\beta_1}\sum (y_ix_i)$$ which is $$L.H.S=\Big[\sum y_i^2-\hat{\beta_0}\sum y_i-\hat{\beta_1}\sum (y_ix_i)\Big]+\sum(\hat{\beta_0}+\hat{\beta_1}x_i)^2-\hat{\beta_0}\sum y_i-\hat{\beta_1}.\sum (y_ix_i).$$ Now what we have inside the bracket is actually the R.H.S. So you need to show that the rest is zero i.e. $$\sum(\hat{\beta_0}+\hat{\beta_1}x_i)^2-\hat{\beta_0}\sum y_i-\hat{\beta_1}\sum (y_ix_i)=0.$$ Now to show this, I will give you some hints. You need to do them correctly and step by step.


*

*Replace $\hat{\beta_0}$ by $\bar{y}-\hat{\beta_1}\bar{x}$ to re-write it all based on $\hat{\beta_1}$.

*Expand the terms and simplify (some terms will be canceled out).

*Now use two facts:
(1): $\hat{\beta_1}=\dfrac{S_{xy}}{S_{xx}},$ where $S_{xy}=\sum(x_i-\bar{x})(y_i-\bar{y})$ and $S_{xx}=\sum(x_i-\bar{x})^2$ and
(2): $S_{xy}=\sum (x_iy_i)-\dfrac{\sum x_i .\sum y_i}{n}$
to write everything in terms of $S_{xy}$ and $S_{xx}$.

*Simplify to show that it is zero.

A: Alternatively, one can show that $$\sum(\hat{\beta_0}+\hat{\beta_1}x_i)^2-\hat{\beta_0}\sum y_i-\hat{\beta_1}\sum y_ix_i=0.$$ by regrouping it to $$\hat{\beta_0} \sum(y_i-\hat{\beta_0}-\hat{\beta_1}x_i)+\hat{\beta_1}\sum (y_i-\hat{\beta_0}-\hat{\beta_1}x_i)x_i=0.$$ and invoking the minimisation argument $$0=\frac{\partial}{\partial{\hat{\beta_0}}}\sum(y_i-\hat{\beta_0}-\hat{\beta_1}x_i)^2=2\sum(y_i-\hat{\beta_0}-\hat{\beta_1}x_i)$$ $$0=\frac{\partial}{\partial{\hat{\beta_1}}}\sum(y_i-\hat{\beta_0}-\hat{\beta_1}x_i)^2=2\sum(y_i-\hat{\beta_0}-\hat{\beta_1}x_i)x_i$$
