In linear regression, the $R^2$ value is the square of the correlation between predicted values and observed values. But why do we need the $R^2$ value? Why not just use the correlation coefficient? Just like the correlation coefficient, $R^2$ is scale-less (i.e. values are always between 0 and 1), so I can't see why there's a need for $R^2$. I would imagine it is something to do with the fact that the correlation coefficient can be negative, but don't really see why this would be a problem.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
8
Since predicted and observed values cannot be negatively correlated, you do not lose any information by squaring the linear correlation coefficient here.
One advantage of the R-squared is its nice objective interpretation as "Proportion of variance (of the response) explained by (differences in) predictors", which follows directly of its definition $$ 1 - \frac{Var(e)}{Var(Y)}, $$ where $e$ is the vector of residuals, $Y$ the vector of observed values of the response and $Var$ is the sample variance. The numerator can be called "Unexplained variance", thus the interpretation above.
-
$\begingroup$ Similar "nice objective interpretations" in terms of other measures of variation would seem to be available, too, such as $1-SD(e)/SD(Y)$. You might therefore want to explain why variance ratios are special or important. $\endgroup$– whuber ♦Aug 29, 2014 at 14:48
-
2$\begingroup$ For univariate regression, it is difficult to think of any real advantage of using $R^2$, because--as the OP notes--that actually loses information about the direction of the relationship afforded by the sign of $r$. Maybe this question should be approached by expanding consideration to the case of multiple regression, where $r$ has no clear analog but $R^2$ remains interpretable. Another way to address these issues is in terms of ANOVA, which seeks to decompose measures of variation into additive components: that practically forces $R^2$ upon us and justifies your answer. $\endgroup$– whuber ♦Aug 29, 2014 at 15:18
-
1$\begingroup$ Just a clarification of @Michael's answer. The quantity $1-Var(e)/Var(Y)$ is the adjusted $R^2$. The ordinary $R^2$ is the same thing with the sums of squares: $R^2=1 - SS(\mbox{error})/SS(\mbox{total})$. $\endgroup$ Aug 29, 2014 at 16:38
-
1$\begingroup$ @RussLenth: For "Var" being the ordinary sample variance with $(n-1)$ in the denominator, "my" definition is equivalent to yours (just divide the two SS by $n-1$). $\endgroup$ Aug 30, 2014 at 13:59
-
1$\begingroup$ @MichaelMayer: But that's not how people compute the variance of the residuals. They use $Var(e)=MS_E=SS_E/(n-p)$, where $p$ is the number of parameters in the regression equation. $\endgroup$ Aug 30, 2014 at 14:07