Given a regression model of the form:

$$y_{i} = \beta_{0} + \beta_{1}x_{i}$$

The least squares estimators for $\beta_{1}$ and $\beta_{0}$ are given by:

$$\hat{\beta}_{1} = \frac{\sum x_{i}y_{i} - n\bar{x}\bar{y}}{\sum x_{i}^{2} - n \bar{x}^{2}}$$


$$\hat{\beta}_{0} = \bar{y} -\hat{\beta}_{1}\bar{x}$$

In addition, $\hat{\beta}_{1}$ and $\hat{\beta}_{0}$ can be expressed in the form:

$$ \hat{\beta}_{1} = \sum_{i=1}^{n}k_{i}y_{i}$$


$$ \hat{\beta}_{0} = \sum_{i=1}^{n}k_{i}'y_{i}$$

where $k_{i} = \frac{(x_{i}-\bar{x})}{S_{XX}}$, $k_{i}' = \frac{1}{n} - \frac{(x_{i}-\bar{x})\bar{x}}{S_{XX}}$ and $S_{XX} = \sum x_{i}(x_{i}-\bar{x})$.

My question is: how do you get from the first set of equations to the second set (functions of $k$ and $k'$) of equations?

Although this may look very like a homework exercise, I can assure you that it's not.


I should probably provide a bit of context here.

Properties such as the expected value and variance of the estimators appear to be more readily obtainable in the forms above containing the $k$ and $k'$ terms. In the event that I want to obtain such properties from alternative regression models, it would seem that it is advantageous to manipulate the estimators' equations until they are in a form comparable to the second set of equations given above.


I'm assuming you know how to derive the estimates $\hat{\beta}_1=\frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^{n}(x_i-\bar{x})^2}$ and $\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}$, which are equivalent to what you've written above.

For $\hat{\beta}_1$:

\begin{eqnarray*} \hat{\beta}_1 &=& \frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^{n}(x_i-\bar{x})^2}\\ &=& \frac{\sum_{i=1}^{n}y_i(x_i-\bar{x})-\bar{y}\sum_{i=1}^{n}(x_i-\bar{x})}{\sum_{i=1}^{n}x_i(x_i-\bar{x})-\bar{x}\sum_{i=1}^{n}(x_i-\bar{x})}\\ &=& \frac{\sum_{i=1}^{n}y_i(x_i-\bar{x})}{\sum_{i=1}^{n}x_i(x_i-\bar{x})}\\ &=& \sum_{i=1}^{n}\frac{(x_i-\bar{x})}{S_{xx}}y_i\\ &=& \sum_{i=1}^{n}k_iy_i \end{eqnarray*}

where $k_i=\frac{(x_i-\bar{x})}{S_{xx}}$.

For $\hat{\beta}_0$:

\begin{eqnarray*} \hat{\beta}_0 &=& \bar{y}-\hat{\beta}_1\bar{x}\\ &=& \frac{1}{n}\sum_{i=1}^{n}y_i-\bar{x}\sum_{i=1}^{n}k_iy_i\\ &=& \sum_{i=1}^{n}\left(\frac{1}{n}-\bar{x}k_i\right)y_i\\ &=& \sum_{i=1}^{n}k_i'y_i\\ \end{eqnarray*}

where $k_i'=\frac{1}{n}-\bar{x}k_i=\frac{1}{n}-\frac{\bar{x}(x_i-\bar{x})}{S_{xx}}$.

| cite | improve this answer | |
  • $\begingroup$ I hadn't realised that $\sum_{i=1}^{n}(x_{i}-\bar{x}) = 0$ until I saw your answer. Thanks a million! $\endgroup$ – user9171 Oct 31 '12 at 10:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.