Deriving Least Squares formula

Question

Working my way through a copy of Draper and Smith's Applied Regression Analysis. I am on the first chapter and looking at the section that involves deriving the OLS equations. No linear algebra has been used yet.

The section I am looking at has started by presenting the sum of squared deviations equation, which has then been differentiated for the intercept and slope parameters and set to 0 to end with the formula for the intercept:

$$\sum_{i=0}^n(Y_i-b_0-b_1X_i)=0$$

And the slope:

$$\sum_{i=0}^nX_i(Y_i-b_0-b_1X_i)=0$$

The next step is to substitute $\beta_0$ for $b_0$ and $\beta_1$ for $b_1$ which apparently leads you to:

$$\sum_{i=0}^nY_i-nb_0-b_1\sum_{i=0}^nX_i=0$$

And

$$\sum_{i=0}^nX_iY_i-b_0\sum_{i=0}^nX_i-b_1\sum_{i=0}^nX_i^2=0$$

I really am at a loss in understanding how the authors got from the first two equations to the second two. They say that they simply substituted $β_0$ for $b_0$ and $β_1$ for $b_1$. Not sure how this changes anything. Can anyone explain?

Answer 1

It's nothing but basic algebra.

For the first equation:

\begin{align*} \sum_{i=1}^n(Y_i-b_0-b_1X_i) &= \sum_{i=1}^nY_i-\sum_{i=1}^nb_0-\sum_{i=1}^nb_1X_i \\ &= \sum_{i=1}^nY_i-nb_0-b_1\sum_{i=1}^nX_i \end{align*}

For the second equation:

\begin{align*} \sum_{i=1}^nX_i(Y_i-b_0-b_1X_i) &= \sum_{i=1}^n(X_iY_i-b_0X_i-b_1X^2_i) \\ &= \sum_{i=1}^nX_iY_i-\sum_{i=1}^n b_0X_i-\sum_{i=1}^n b_1X^2_i \\ &= \sum_{i=1}^nX_iY_i-b_0\sum_{i=1}^n X_i-b_1\sum_{i=1}^n X^2_i \end{align*}

$b_0$ and $b_1$ don't depend on $i$ can be pulled out of the summation via the distributive property.

For example:

\begin{align*} \sum_{i=1}^n b x_i &= b x_1 + b x_2 + b x_3 + \ldots + b x_n \\ &= b\left( x_1 + x_2 + x_3 + \ldots + x_n \right)\\ &= b \sum_{i=1}^n x_i \end{align*}