3
$\begingroup$

The R2 of a simple linear regression model is the squared Pearson correlation coefficient (r) between the observations and the fitted values.

Isn't the above in contradiction with the fact that the R2 of a simple linear regression model is negative when the model fits the data worse than a horizontal line? That is, when the model predictions do not follow the trend of the data, as shown in the "wrong model" case below? In that case, the R2 is clearly not equal to the squared Pearson coefficient (nor to the square of anything), since it is negative.

Also, the $r=-0.96$ in the "wrong model" case below is misleading to me. Without looking at the plot, I would have interpreted it as an indication of a strong negative correlation (like the first plot, but going downward), whereas we can see that there is no correlation at all.

Any tips? Am I missing something?

EDIT according to this discussion, the above is possible if the model is constrained, which seems to answer part of my question. The $r=-0.96$ in the third plot is still misleading to me though.

enter image description here

Notes:

  1. In the baseline case, r is technically undefined as the standard deviation is null. But clearly, is a variable is constant, it cannot be correlated with any other variable.
  2. The definition of the R2 I am using is:
my_r2 = function(y,y_hat){
  1 - sum((y-y_hat)^2)/sum((y-mean(y))^2)
}

For Pearson, I am using R's cor() function with the default arguments.

$\endgroup$
4
  • 1
    $\begingroup$ "Simple linear regression" always includes an intercept. Such a model can never have an $R^2$ less than zero. $\endgroup$
    – whuber
    May 25 '21 at 17:00
  • $\begingroup$ Yes, except if the model is "constrained", i.e., if its parameters (intercept and/or slope) are specified by hand instead of being fitted via OLS (see the discussion linked above). The other case when the $R^2$ can be negative is when the adjusted $R^2$ is used. $\endgroup$
    – Antoine
    May 25 '21 at 17:49
  • 1
    $\begingroup$ Sure: when you change the meaning of "simple linear regression" or the meaning of "R^2," standard results no longer hold! These issues are extensively discussed elsewhere on CV: see Adjusted R and negative R square. $\endgroup$
    – whuber
    May 25 '21 at 18:26
  • 1
    $\begingroup$ Yes yes, I agree! $\endgroup$
    – Antoine
    May 25 '21 at 18:30
3
$\begingroup$

You are correct in saying that R2 can be negative, and in concluding that it is not in general the square of Pearson's correlation, or of any other real statistic.

If the model has been made by least-squares regression, then it is true that R2 is the square of the correlation coefficient. Least squares regression does not always make a good model, but it is never worse than the baseline prediction of the mean value throughout. (To be more exact, it is never worse if squared error is the appropriate loss.)

$\endgroup$
1
  • 2
    $\begingroup$ You should add "... and the model includes an intercept" $\endgroup$ May 25 '21 at 16:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.