1
$\begingroup$

How can I write the sum of squared residuals as a function of the sample mean and variance of $y$, given that the regression equation is:

$y = \beta_0 + \beta_1(x-\bar{x}) + \epsilon$

where $\bar{x}$ is the average of the $x$'s.

I want to calculate the sum of the squared residuals using only the 1st and 2nd sample moments of $x$ and $y$.

$\endgroup$
8
  • 1
    $\begingroup$ If this is for some course, please include the self-study tag. $\endgroup$
    – Glen_b
    Sep 15 '13 at 6:26
  • 1
    $\begingroup$ A small hint: what's $\hat \beta_0$? $\endgroup$
    – Glen_b
    Sep 15 '13 at 6:26
  • $\begingroup$ well its the average of the y values, but I don't see how that helps... $\endgroup$ Sep 15 '13 at 10:53
  • $\begingroup$ @ Glen_b I mean I still have the b1_hat floating around. I guess if I know b1_hat i could plug values in after expanding the expression but it would not be very nice. Sorry I am just not seeing it even though I actually spent about 3 hours today trying some different, and probably very wrong, methods. $\endgroup$ Sep 15 '13 at 11:10
  • $\begingroup$ AH! I just got it sorry I was being silly. Thanks for the hint Glen_b :) $\endgroup$ Sep 15 '13 at 11:29
1
$\begingroup$

Here is my deduction, hope it helps.

Given the regression equation,

\begin{equation} \hat{y} = \beta_0 + \beta_1(x - \overline{x}) + \epsilon \end{equation}

By minimizing the sum of squared errors, we can get

\begin{equation} \hat{\beta_0} = \overline{y}\\ \hat{\beta_1} = \frac{Cov(x, y)}{Var(x)} \end{equation}

Then, assuming there are n samples, we can calculate the sum of squared residuals

\begin{eqnarray*} RSS&=&\sum_{i}(y_i - \hat{y_i})^2\\ &=&\sum_{i}(y_i - \hat{\beta_0} - \hat{\beta_1}(x_i - \overline{x}))^2\\ &=&\sum_{i}(y_i - \overline{y} - \hat{\beta_1}(x_i - \overline{x}))^2\\ &=&\sum_{i}(y_i - \overline{y})^2 + \hat{\beta_1}^2\sum_{i}(x_i - \overline{x})^2 -2\hat{\beta_1}\sum_{i}(x_i - \overline{x})(y_i - \overline{y})\\ &=&n(Var(y) - \frac{Cov(x,y)^2}{Var(x)}) \end{eqnarray*}

$\endgroup$
1
  • $\begingroup$ You could write that last line also as: $$n(1-\rho^2) Var(y)$$ which relates it to $R^2 = 1- SS_{res}/SS_{tot}$ or differently written: $$SS_{res} = SS_{tot} (1-R^2) = nVar(y) (1-R^2)$$ $\endgroup$ Jan 19 '19 at 9:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.