# Why define coefficient of determination as 1 - RSS/TSS?

Based on wiki, the definition of coefficient of determination is defined as $$1 - RSS/TSS$$, where RSS is the residual sum of squares ($$\sum(y-\hat{y})^2$$) and TSS is the total sum of squares ($$\sum(y-\bar{y})^2$$), with $$\hat{y}$$ the model prediction and $$\bar{y}$$ the sample mean.

It can be shown that $$1 - RSS/TSS = ESS/TSS$$, where ESS is the explained sum of squares ($$\sum(\hat{y}-\bar{y})^2$$). It is remarked that only in some cases such as simple linear regression with intercept (are there other cases?), this equality holds.

The right hand side (ESS/TSS) is obviously positive, but the left hand size is not guaranteed to be positive. That is why in some cases we see negative coefficient of determinations. However, almost all the sources I read online define coefficient of determination as $$1 - RSS/TSS$$.

My question is that, why not define coefficient of determination as $$ESS/TSS$$ instead? In this way it is always positive.

My attempt. I guess that the disadvantage of the alternative definition $$ESS/TSS$$ is that it can be larger than 1. (The advantage of the normal definition $$1-RSS/TSS$$ is that it is always less than 1 as it is 1 subtracts something positive). So I compute $$TSS - ESS = \sum_i (y_i - \bar{y})^2 - \sum_i (\hat{y}_i - \bar{y})^2 \\ = \sum_i (y_i^2 - 2\bar{y}(y_i - \hat{y}_i) - \hat{y}_i^2).$$ Assuming

1. the mean of the errors is zero (so that sample mean is the same as mean of predicted values) and
2. the error has zero correlation with the predicted value, we then have $$TSS - ESS = \sum_i ((y_i - \hat{y}_i)^2 + 2(\hat{y}_i - \bar{{y}})(y_i - \hat{y}_i))\\ = RSS + C = RSS,$$ where the term $$C$$ is proportional to the correlation $$Corr(\hat{y}, \epsilon)$$, where $$\epsilon_i = y_i - \hat{y}_i$$. It seems we always have $$TSS = ESS + RSS$$ suppose the two assumptions hold. So, is it correct that the two assumptions 1 and 2 are so strong that they are often not satisfied so that $$ESS/TSS > 1$$?
• If it is an equality - which holds for any linear regression - it is impossible that the rhs is always positive and the lhs possibly not! The reason for a possible negative R^2 in whatever definition is when no intercept (or some linear combination of regressors for an intercept) is included in the regression Mar 21 at 10:19
• You're missing several square superscripts early on and there may be other typos. I'll not edit for fear that typographical corrections will interfere with clarifying whatever is concerning or confusing you. Mar 21 at 10:24
• Mar 21 at 10:26
• @ChristophHanck, in the case where the intercept is missing in a linear regression model, we no longer have the assumption 1 and hence the equality is not satisfied. Mar 21 at 12:20
• That is correct - but what I am trying to say is that if you have an intercept, you cannot have a negative R^2 in either (equivalent) formulation. Also note that this is not really an assumption, but an algebraic property of OLS Mar 21 at 16:17

## 3 Answers

It can be shown that $$1 - RSS/TSS = ESS/TSS$$, .... It is remarked that only in some cases such as simple linear regression with intercept (are there other cases?), this equality holds.

This is not true. Remaining in the realm of linear regression this equality holds quite generally. Indeed it holds in any multiple regression if the constant is included, not only in the simple (one regressor) case. In this setting $$R^2$$ is bounded between $$0$$ and $$1$$ and such a thing is good even to facilitate communications, especially among non-specialist people.

The right hand side (ESS/TSS) is obviously positive, but the left hand size is not guaranteed to be positive. That is why in some cases we see negative coefficient of determinations. However, almost all the sources I read online define coefficient of determination as $$1 - RSS/TSS$$.

My question is that, why not define coefficient of determination as $$ESS/TSS$$ instead? In this way it is always positive.

The presence of the constant is important and involves the definition and properties of: TSS, ESS, RSS. See here https://math.stackexchange.com/questions/2398194/why-is-r-square-not-well-defined-for-a-regression-without-a-constant-term

Note that if the intercept is not included It is usual to re-open discussion of the definitions. Then it is true that we can achieve forms similar to:

$$R^2 = 1-RSS/TSS$$

or

$$R^2 = ESS/TSS$$

but the two are no longer equal in general.

So, as you said, in such a situation it can be shown that the form like $$R^2 = 1-RSS/TSS$$ can return negative values. However, it can be shown that even the form like $$R^2=ESS/TSS$$ returns unbounded values: indeed, they can be greater than $$1$$ (for some insight read here: Can $R^2$ be greater than 1?). So your proposal is not a panacea: in both cases the readings is not so easy.

Now I see that you see the drawbacks of the form $$ESS/TSS$$. So you can simply consider that the form $$1-RSS/TSS$$ is preferable because it explicitly considers residuals, a desirable property for a definition of a fitting measure.

This may be tangential to the main thrust of this question, but I think it deserves saying.

I wouldn't define $$R^2$$ in this way at all, although which formula is best for calculation in which circumstances remains a distinct question that is still of importance in its own way.

In my view, it's simplest and best to regard $$R^2$$ as the square of the Pearson correlation, as the notation implies. This has several advantages, mostly very simple, but that simplicity makes them cogent and attractive:

1. Pedagogy and motivation The measure can be introduced as a relative of Pearson correlation whenever -- as happens often but not always -- correlation has been introduced to people earlier. Usually, $$R^2$$ will be explained in the context of the Pearson correlation $$r$$ for two variables $$x, y$$ for which a line $$\hat y = a + bx$$ is fitted by regression.

2. Applicability across regression The idea applies across regression with several predictors and to nonlinear regression. where the interpretation that $$R^2$$ is $$\text{corr}(y, \hat y)^2$$ can be maintained.

3. Applicability even more widely The same idea can also be applied to other models. For example, Zheng and Agresti in 2000 advocated the use of similar ideas across generalized linear models.

Zheng, B. and Agresti, A. 2000, Summarizing the predictive power of a generalized linear model. Statistics in Medicine 19: 1771-1781. https://doi.org/10.1002/1097-0258(20000715)19:13<1771::AID-SIM485>3.0.CO;2-P

There are many natural qualifications here on various levels. As you move away from linear regression, the case for using other single-value figures of merit of goodness or badness of fit or other dimensions of model performance is often stronger and may even seem compelling. It is easy to think of set-ups with very high $$R^2$$ that have little scientific or statistical value and conversely of set-ups where very low $$R^2$$ is expected and associated with an interesting model.

But in terms of the question: any formulation in terms of sums of squares is in danger of not seeing the wood for the trees.

• But Pearson correlation coefficient describes only linear correlation. The correlation coefficient may not be useful when we do a simple linear regression with higher order terms? Mar 21 at 12:05
• Sure, but note the definition of correlation as given here. It is the correlation between observed and predicted by a model, which should always be defined by a single value for each observation. Mar 21 at 12:56
• could you elaborate more on "which should always be defined by a single value for each observation". In my understanding of Pearson correlation, you still need a sample of two random variables to estimate it, isn't it? Moreover, $Corr(y, \hat{y}) = Corr(y, a+bx) = Corr(y, x)$. I don't get it how "It is the correlation between observed and predicted by a model" makes any difference? thank you Mar 21 at 15:51
• Put $\hat y = Xb$ for predictors $X$ and coefficient estimates $b$. There is still one, and only one, value of $\hat y$ for each observation. And so on. , Mar 21 at 17:24
1. You can define, calculate, and report anything you want. If you think it would be helpful to report $$ESS/TSS$$, go for it! If you make a compelling case, then you’ve got a seminal paper that everyone in your field will cite, and you just locked up tenure.

(Maybe that last part is overly optimistic.)

1. I see $$1-RSS/TSS$$ as a comparison of model performance. The numerator is the performance of your model as measured by square loss. The denominator is the square loss of a reasonable baseline “must beat” model that estimates the conditional mean to be the marginal mean, $$\bar y$$, no matter what the feature values are. Thus, this calculation can be seen as measuring how much of the garbage (loss) has been taken out. If you take out $$30\%$$ of the garbage, that is better than $$20\%$$ but worse than $$40\%$$. Such a calculation also gives a flag for performance worse than the naïve benchmark model when scores are below zero.

2. The statistic can be written as $$(TSS-RSS)/TSS$$, which is a percent change. While total sum of squares is not equal to the sum of the residual and “explained” sums of squares in all cases, so I am hesitant to always refer to $$1-RSS/TSS$$ as the proportion of variance explained, such a calculation clearly has an interpretation as the percent reduction of something undesirable.

3. Sure, one can always just compare the model performance to the performance of a benchmark, but this calculation forces that to be considered. The analyst can’t just say, “Square loss looks small.” There is some context given to that value of square loss.

Here, I give additional thoughts on calculating and interpreting $$R^2$$, and a citation at the end shows that my stance is supported by modern statistics literature. I will give a few more links below.

What does "explained variation" mean in reference to R-squared?

Is there such a thing as a too low R-squared when running multiple linear regression?

Relationship between R^2 and sum of squared errors in non-linear models

These final five links deal with generalizations.

Quantile Regression Pseudo R-Squared

Which metric to use to evaluate Quantile Regression?

What sense does adjusted $$R^2$$ and deviance explained mean for quantile generalized additive models (QGAMs)?

Is the proportion classified correctly a reasonable analogue of $$R^2$$ for a classification model?

How arbitrary is adjusted r-squared as a measure of fit?

• Thank you for the nice answer. Indeed, $1−RSS/TSS$ is a percent reduction of a benchmark error and is not necessarily the proportion of variance explained. I never notice that sklearn used out-sample mean in the denominator of the formula of r2. I agree that fixing the mean as in-sample mean seems to make more sense! Mar 21 at 12:18