R2 and correlation coefficient relationship - Cross Validated most recent 30 from stats.stackexchange.com 2019-07-17T11:03:14Z https://stats.stackexchange.com/feeds/question/316633 http://www.creativecommons.org/licenses/by-sa/3.0/rdf https://stats.stackexchange.com/q/316633 1 R2 and correlation coefficient relationship curio1729 https://stats.stackexchange.com/users/82443 2017-12-01T11:01:20Z 2017-12-03T17:12:03Z <p>R2 score can be negative. Then how is R2 square of correlation coefficient? </p> https://stats.stackexchange.com/questions/316633/r2-and-correlation-coefficient-relationship/316652#316652 2 Answer by Ruben van Bergen for R2 and correlation coefficient relationship Ruben van Bergen https://stats.stackexchange.com/users/38185 2017-12-01T13:14:01Z 2017-12-01T13:14:01Z <p>\$R^2\$ can only be negative if your model (erroneously) doesn't include an intercept, or if you're computing it on a data set independent to the data that the model was estimated on, and the model doesn't generalize well to that new data set.</p> <p>A correlation coefficient, between two variables \$(x,y)\$, basically describes the goodness-of-fit of a simple linear model, consisting of a single predictor (\$x\$) and an intercept. The way the correlation coefficient is computed is equivalent to fitting this linear model to some data, and computing its goodness-of-fit on the same data. Therefore, the correlation coefficient does not meet either of the criteria above, and thus \$R^2\$ values derived from correlation coefficients are positive by definition.</p> <p>Another way to see this is that at worst, a correlation coefficient can be 0, which means the univariate linear model implied in the correlation analysis explains no variance. Explaining no variance is equivalent to an \$R^2\$ of 0. But it is impossible for the correlation model to explain negative variance, i.e. to perform worse than the sample mean, since the correlation model includes the sample mean through its intercept.</p> <p>For a correlation analysis, \$R^2\$ can be computed as the square of the correlation coefficient, and will then behave as described above. But as Ami Tavory pointed out, \$R^2\$ has other definitions, and in general, can be computed for any model, not just correlations. In those cases, one of the scenarios may occur that I described at the beginning, and then \$R^2\$ can be negative.</p> https://stats.stackexchange.com/questions/316633/r2-and-correlation-coefficient-relationship/316846#316846 0 Answer by kjetil b halvorsen for R2 and correlation coefficient relationship kjetil b halvorsen https://stats.stackexchange.com/users/11887 2017-12-02T19:57:24Z 2017-12-02T19:57:24Z <p>It is probably because you are looking at an adjusted R-squared, not the R-squared itself. You can find definition and a formula here: <a href="https://en.wikipedia.org/wiki/Coefficient_of_determination#Adjusted_R2" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Coefficient_of_determination#Adjusted_R2</a> </p> <p>The idea is that R-squared will always increase in you include more variables in your model, even if the new variables you include do not have any real predictive power. The adjusted R-squared is a way to try to make a correction to that effect, and a side-effect of the correction is that it sometimes gives a negative value. </p>