# Least Squares Estimators' Forms

Given a regression model of the form:

$$y_{i} = \beta_{0} + \beta_{1}x_{i}$$

The least squares estimators for $\beta_{1}$ and $\beta_{0}$ are given by:

$$\hat{\beta}_{1} = \frac{\sum x_{i}y_{i} - n\bar{x}\bar{y}}{\sum x_{i}^{2} - n \bar{x}^{2}}$$

and

$$\hat{\beta}_{0} = \bar{y} -\hat{\beta}_{1}\bar{x}$$

In addition, $\hat{\beta}_{1}$ and $\hat{\beta}_{0}$ can be expressed in the form:

$$\hat{\beta}_{1} = \sum_{i=1}^{n}k_{i}y_{i}$$

and

$$\hat{\beta}_{0} = \sum_{i=1}^{n}k_{i}'y_{i}$$

where $k_{i} = \frac{(x_{i}-\bar{x})}{S_{XX}}$, $k_{i}' = \frac{1}{n} - \frac{(x_{i}-\bar{x})\bar{x}}{S_{XX}}$ and $S_{XX} = \sum x_{i}(x_{i}-\bar{x})$.

My question is: how do you get from the first set of equations to the second set (functions of $k$ and $k'$) of equations?

Although this may look very like a homework exercise, I can assure you that it's not.

EDIT:

I should probably provide a bit of context here.

Properties such as the expected value and variance of the estimators appear to be more readily obtainable in the forms above containing the $k$ and $k'$ terms. In the event that I want to obtain such properties from alternative regression models, it would seem that it is advantageous to manipulate the estimators' equations until they are in a form comparable to the second set of equations given above.

I'm assuming you know how to derive the estimates $\hat{\beta}_1=\frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^{n}(x_i-\bar{x})^2}$ and $\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}$, which are equivalent to what you've written above.

For $\hat{\beta}_1$:

\begin{eqnarray*} \hat{\beta}_1 &=& \frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^{n}(x_i-\bar{x})^2}\\ &=& \frac{\sum_{i=1}^{n}y_i(x_i-\bar{x})-\bar{y}\sum_{i=1}^{n}(x_i-\bar{x})}{\sum_{i=1}^{n}x_i(x_i-\bar{x})-\bar{x}\sum_{i=1}^{n}(x_i-\bar{x})}\\ &=& \frac{\sum_{i=1}^{n}y_i(x_i-\bar{x})}{\sum_{i=1}^{n}x_i(x_i-\bar{x})}\\ &=& \sum_{i=1}^{n}\frac{(x_i-\bar{x})}{S_{xx}}y_i\\ &=& \sum_{i=1}^{n}k_iy_i \end{eqnarray*}

where $k_i=\frac{(x_i-\bar{x})}{S_{xx}}$.

For $\hat{\beta}_0$:

\begin{eqnarray*} \hat{\beta}_0 &=& \bar{y}-\hat{\beta}_1\bar{x}\\ &=& \frac{1}{n}\sum_{i=1}^{n}y_i-\bar{x}\sum_{i=1}^{n}k_iy_i\\ &=& \sum_{i=1}^{n}\left(\frac{1}{n}-\bar{x}k_i\right)y_i\\ &=& \sum_{i=1}^{n}k_i'y_i\\ \end{eqnarray*}

where $k_i'=\frac{1}{n}-\bar{x}k_i=\frac{1}{n}-\frac{\bar{x}(x_i-\bar{x})}{S_{xx}}$.

• I hadn't realised that $\sum_{i=1}^{n}(x_{i}-\bar{x}) = 0$ until I saw your answer. Thanks a million! – user9171 Oct 31 '12 at 10:00