14
$\begingroup$

This may be a basic question, but I was wondering why an $R$ value in a regression model can simply be squared to give a figure of explained variance?

I understand that $R$ coefficient can give the strength of a relationship, but I don't understand how simply squaring this value gives a measure of explained variance.

Any easy explanation of this?

Thanks very much for helping with this!

$\endgroup$
  • $\begingroup$ Are you looking for something intuitive or more mathematical? Have you looked through some of the other questions on $R^2$ and correlation coefficients on this site? $\endgroup$ – cardinal May 9 '12 at 23:46
  • 1
    $\begingroup$ Two related questions are here and here, for example. If you play around with the equations there, you'll be able to derive the mathematical equivalence. But, neither are likely to be particularly helpful from an intuition standpoint. $\endgroup$ – cardinal May 9 '12 at 23:47
  • $\begingroup$ I see this the opposite way. It is R square that is defined as 1 -residual variance/total variance and then R is postive square root of that. It just happens that when we have simple linear regression R square reduces to the square of the correlation coefficient. $\endgroup$ – Michael R. Chernick May 10 '12 at 0:37
  • $\begingroup$ @Michael, you undoubtedly intended to say the appropriately signed square root rather than the positive one. $\endgroup$ – cardinal May 10 '12 at 10:28
  • 1
    $\begingroup$ @cardinal, I have the same impression - $R$ (or $r$) refers to the sample correlation coefficient and would be surprised to see a widely used reference that uses that to refer to the absolute value of the sample correlation $\endgroup$ – Macro May 10 '12 at 12:39
16
$\begingroup$

Hand-wavingly, the correlation $R$ can be thought of as a measure of the angle between two vectors, the dependent vector $Y$ and the independent vector $X$. If the angle between the vectors is $\theta$, the correlation $R$ is $\cos(\theta)$. The part of $Y$ that is explained by $X$ is of length $||Y||\cos(\theta)$ and is parallel to $X$ (or the projection of $Y$ on $X$). The part that is not explained is of length $||Y||\sin(\theta)$ and is orthogonal to $X$. In terms of variances, we have $$\sigma_Y^2 = \sigma_Y^2\cos^2(\theta) + \sigma_Y^2\sin^2(\theta)$$ where the first term on the right is the explained variance and the second the unexplained variance. The fraction that is explained is thus $R^2$, not $R$.

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ (+1) Not too much handwaving going on here really. The geometric viewpoint is the most intuitive, in my view. There is likely to be a high-quality open-source figure out there that depicts things precisely this way. $\endgroup$ – cardinal May 10 '12 at 10:25
  • $\begingroup$ (+1) I started to write up a direct derivation that ${\rm cor}(y,\hat{y})^2$ was equal to the usual definition of $R^2$ as a ratio of variances but, in doing so, I noticed it provided little/no intuition (and so it probably wouldn't be helpful to the original poster) - I think this does! $\endgroup$ – Macro May 10 '12 at 12:17
  • 1
    $\begingroup$ This doesn't answer the question but shows how R square is mentioned as the square of the correlation coefficient without any reference to R. So sources confirming or refuting my claim may be hard to find. This is from an article on the coefficient of determination in Wikipedia: $\endgroup$ – Michael R. Chernick May 10 '12 at 14:55
  • $\begingroup$ As squared correlation coefficient Similarly, after least squares regression with a constant+linear model (i.e., simple linear regression), R2 equals the square of the correlation coefficient between the observed and modeled (predicted) data values. $\endgroup$ – Michael R. Chernick May 10 '12 at 14:56
  • $\begingroup$ Under general conditions, an R2 value is sometimes calculated as the square of the correlation coefficient between the original and modeled data values. In this case, the value is not directly a measure of how good the modeled values are, but rather a measure of how good a predictor might be constructed from the modeled values (by creating a revised predictor of the form α + βƒi). According to Everitt (2002, p. 78), this usage is specifically the definition of the term "coefficient of determination": the square of the correlation between two (general) variables. $\endgroup$ – Michael R. Chernick May 10 '12 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.