1
$\begingroup$

I understand, mathematically, how to get the estimates of coefficients in ordinary least squares. What I am struggling with is coming up with a conceptual, geometric explanation for the correlation part of the simple linear regression coefficient.

Say we have the following equation: $$ y = \hat\beta_1x + \hat\beta_0 $$ where $$ \hat\beta_1 = \frac{S_{xy}}{S^2_x} = r_{xy}\frac{S_y}{S_x} $$ and $$ \hat\beta_0 = \overline{y} - \hat\beta_1\overline{x} $$

If I understand correctly, we are essentially just getting estimates of spread ($\hat\beta_1$) and position ($\hat\beta_0$) shifts that can turn $x$ into $y$ as best as possible (least squares). So from what I understand, the ratio of standard deviations ($\frac{S_y}{S_x}$) makes it so $x$ has the same spread as $y$, and then the estimate of $\beta_0$ moves $x$ to the same position as $y$.

What I am missing is what the correlation $r_{xy}$ is doing in this linear transformation. If we have shifted the spread and position of $x$ as above, what is $r_{xy}$ representing?

$\endgroup$
1
$\begingroup$

$r_{xy}$ is the slope of the OLS regression line in that transformed space. You can also think of $r^2_{xy}$ as the proportion of the variance in the transformed $y$ variable that is explainable by virtue of knowledge of the transformed $x$, and $1-r^2_{xy}$ as the error variance, or the variance that cannot be so explained.

$\endgroup$
  • $\begingroup$ Oh wow okay that makes total sense - I don't know why I couldn't get there. Thanks! $\endgroup$ – RickyB Jul 29 '13 at 15:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.