Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Correlation, $r$, is a measure of linear association between two variables. Coefficient of determination, $r^2$, is a measure of how much of the variability in one variable can be "explained by" variation in the other.

For example, if $r = 0.8$ is the correlation between two variables, then $r^2 = 0.64$. Hence, 64% of the variability in one can be explained by differences in the other. Right?

My question is, for the example stated, is either of the following statements correct?

  1. 64% of values fall along the regression line
  2. 80% of values fall along the regression line
share|improve this question
up vote 5 down vote accepted

The first part of this is basically correct - but it's 64% of the variation is explained by the model. In a simple linear regression: Y ~ X, if $R^2$ is .64 it means that 64% of the variation in Y is determined by the linear relationship between Y and X. It is possible to have a strong relationship with very low $R^2$, if the relationship is strongly non-linear.

Regarding your two numbered questions, neither is correct. Indeed, it is possible that none of the points may lie exactly on the regression line. That's not what's being measured. Rather, it is a question of how close the average point is to the line. If all or nearly all points are close (even if none are exactly on the line) then $R^2$ will be high. If most points are far from the line, $R^2$ will be low. If most points are close but a few are far, then the regression is incorrect (problem of outliers). Other things can go wrong, too.

In addition, I've left the notion of "far" rather vague. This will depend on how spread out the X's are. Making these notions precise is part of what you learn in a course on regression; I won't get into it here.

share|improve this answer
    
Well that cleared up a lot for me! Thank you Mimshot and Peter Flom! Much grateful to you both! :) – Bradex Dec 8 '12 at 15:50
1  
+1, good answer, would you mind adding something like "Indeed, [it is possible that] none of the points may lie... ". Also, it might be worth discussing that the notion of how far the points are from the line is also relative to how spread out the X's are. – gung Dec 8 '12 at 15:56

You are right with the first part of your statement. The usual way of interpreting the coefficient of determination $R^{2}$ is as the percentage of the variation of the dependent variable $y$ ($Var(y)$) that we are able to explain with the explanatory variables. The exact interpretation and derivation of the coefficient of determination $R^{2}$ can be found here

http://economictheoryblog.com/2014/11/05/the-coefficient-of-determination-latex-r2/

However, a way less known interpretation of the coefficient of determination $R^{2}$ is to interpret it as as the Squared Pearson Correlation Coefficient between the observed values $y_{i}$ and the fitted values $\hat{y}_{i}$. The proof that the coefficient of determination is the equivalent of the Squared Pearson Correlation Coefficient between the observed values $y_{i}$ and the fitted values $\hat{y}_{i}$ can be found here

http://economictheoryblog.com/2014/11/05/proof/

In my view are these the only meaningful ways of interpreting the coefficient of determination $R^{2}$. It follows that the two statements you made cannot be derived from the $R^{2}$.

share|improve this answer
2  
I am not sure there are only two ways to interpret $R^2$ (there are certainly more than two ways to interpret $r$) but the reason it follows that the two statements given can't be derived from the $R^2$ is that they are false (for the reasons @PeterFlom explains) rather than no other interpretation being possible. But I think otherwise this is a nice answer. – Silverfish Oct 11 '15 at 13:02
2  
In case the given links go dead at some point in the future (linkrot is an eternal problem - we prefer making answers self-contained if possible, but clearly this question does not call for full proofs so a link is expedient), we have some coverage of the relationship between $\operatorname{Corr}(y, \hat y)$ and $R^2$, here, here, here and more geometrically, here. – Silverfish Oct 11 '15 at 13:08

Niether 1 nor 2 is correct.

Let's say you are trying to predict a set of values $\pmb{y}$ from a set of values $\pmb{x}$ using a linear regression. Your model is

$$y_i = b + mx_i + \epsilon_i$$

Where $\epsilon_i \sim \mathcal{N(0,\sigma^2)}$ is some noise. $R^2=.64$ means that 64% of the variance of $y$ can be explained by variability in $x$ under your model. The residual variance (i.e., the variance unexplained) is 0.36. That is, if:

$$\hat{y}_i = b + mx_i$$

Then

$$1-0.64 = 0.36 = \frac{\mathrm{var}(\pmb{y}-\pmb{\hat{y}})}{\mathrm{var}(\pmb{y})} $$

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.