4
$\begingroup$

$R^2$ equals the "amount of variance explained by the model".

However, we rarely use variance in descriptive statistics. We say a sample's weight is 78 ± 13 kg, which is $\bar x$ ± $\sigma$ (stdev), not $\bar x$ ± $\sigma^2$ (variance). This is because the standard deviation is in the same scale as the variable and is easier to understand than variance.

Thus my question: why don't we use "amount of standard deviation explained by the model"? Wouldn't you get it with $\sqrt{R^2}$? In my opinion, this would be much more intuitive to understand.

$\endgroup$
6
  • $\begingroup$ I think because $R^2$ is nicely differentiable and more easy to deal with instead of $R$. $\endgroup$
    – Allan
    Aug 4 at 18:08
  • $\begingroup$ @Allan What does "more easy to deal with" mean? Also: Welcome, J. Park! $\endgroup$
    – Alexis
    Aug 4 at 18:31
  • $\begingroup$ @Alexis, I mean that is differentiable, so if use want to use $R^2$ a an objective function is more easily to compute the derivatives and apply numerical methods, for example, gradient descent. $\endgroup$
    – Allan
    Aug 4 at 18:39
  • 1
    $\begingroup$ Variances add; standard deviations don't. $\endgroup$
    – whuber
    Aug 4 at 18:57
  • 1
    $\begingroup$ @Bernhard Right--that's one of the implications. $\endgroup$
    – whuber
    Aug 4 at 19:38

1 Answer 1

1
$\begingroup$

$R^2$ has several equivalent definitions. One is the squared correlation between the $x$ and $y$ variables in a simple linear regression. One is the squared correlation between the true values of $y$ and the predicted values; this has an advantage of working for multiple linear regression.

The definition that seems best-motivated to me involves the decomposition of the total sum of squares (SST) into the regression sum of squares (SSReg) and the residual sum of squares (SSRes). This $1-\frac{SSRes}{SST}$ expression can be considered on its own without thinking about the above two squared correlations. I think of that as a comparison of the performance of the fitted model to a naïve model that always predicts $\bar y$, no matter the feature values. Perhaps we could write $C=1-\frac{SSRes}{SST}$. However, it turns out that $C$ is equal to the above two squared correlations, so it has been given the name $R^2$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.