1
$\begingroup$

How can I write the sum of squared residuals as a function of the sample mean and variance of $y$, given that the regression equation is:

$y = \beta_0 + \beta_1(x-\bar{x}) + \epsilon$

where $\bar{x}$ is the average of the $x$'s.

I want to calculate the sum of the squared residuals using only the 1st and 2nd sample moments of $x$ and $y$.

$\endgroup$
8
  • 1
    $\begingroup$ If this is for some course, please include the self-study tag. $\endgroup$
    – Glen_b
    Commented Sep 15, 2013 at 6:26
  • 1
    $\begingroup$ A small hint: what's $\hat \beta_0$? $\endgroup$
    – Glen_b
    Commented Sep 15, 2013 at 6:26
  • $\begingroup$ well its the average of the y values, but I don't see how that helps... $\endgroup$ Commented Sep 15, 2013 at 10:53
  • $\begingroup$ @ Glen_b I mean I still have the b1_hat floating around. I guess if I know b1_hat i could plug values in after expanding the expression but it would not be very nice. Sorry I am just not seeing it even though I actually spent about 3 hours today trying some different, and probably very wrong, methods. $\endgroup$ Commented Sep 15, 2013 at 11:10
  • $\begingroup$ AH! I just got it sorry I was being silly. Thanks for the hint Glen_b :) $\endgroup$ Commented Sep 15, 2013 at 11:29

1 Answer 1

1
$\begingroup$

Here is my deduction, hope it helps.

Given the regression equation,

\begin{equation} \hat{y} = \beta_0 + \beta_1(x - \overline{x}) + \epsilon \end{equation}

By minimizing the sum of squared errors, we can get

\begin{equation} \hat{\beta_0} = \overline{y}\\ \hat{\beta_1} = \frac{Cov(x, y)}{Var(x)} \end{equation}

Then, assuming there are n samples, we can calculate the sum of squared residuals

\begin{eqnarray*} RSS&=&\sum_{i}(y_i - \hat{y_i})^2\\ &=&\sum_{i}(y_i - \hat{\beta_0} - \hat{\beta_1}(x_i - \overline{x}))^2\\ &=&\sum_{i}(y_i - \overline{y} - \hat{\beta_1}(x_i - \overline{x}))^2\\ &=&\sum_{i}(y_i - \overline{y})^2 + \hat{\beta_1}^2\sum_{i}(x_i - \overline{x})^2 -2\hat{\beta_1}\sum_{i}(x_i - \overline{x})(y_i - \overline{y})\\ &=&n(Var(y) - \frac{Cov(x,y)^2}{Var(x)}) \end{eqnarray*}

$\endgroup$
1
  • $\begingroup$ You could write that last line also as: $$n(1-\rho^2) Var(y)$$ which relates it to $R^2 = 1- SS_{res}/SS_{tot}$ or differently written: $$SS_{res} = SS_{tot} (1-R^2) = nVar(y) (1-R^2)$$ $\endgroup$ Commented Jan 19, 2019 at 9:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.