I'm quite new to this field and would appreciate your help!

As mentioned in this thread, here are the equations for R² and Q²:

$R² = 1 - (RSS / TSS)$

where: $RSS = \sum (y-ŷ)²$

where: $TSS = \sum(y-\bar{y})²$

While Q² is calculated as follows:

$Q² = 1 - (PRESS / TSS)$

where: $PRESS = \sum (y-ŷ_{ext})²$

The Calculation for R2 and Q2 are almost identical, with the only difference being that RSS is calculated from the data on which the algorithm is trained and PRESS is calculated from held out data.

My question is concerning TSS: Is $\bar{y}$ the mean value calculated from the training set in both situations (R² and Q²)? Otherwise, which mean value is it?

And a side question is: Can Q² be higher than R²? and if so, what does it mean?

  • 1
    $\begingroup$ Your equations for RSS and PRESS are identical. Also, note that you can use MathJaX for formulas. $\endgroup$ Commented Dec 14, 2021 at 9:17
  • $\begingroup$ @StephanKolassa! I edited the post to show the difference between RSS and PRESS. And thanks for recommending MathJax, I will check it out right away! $\endgroup$
    – as_odai
    Commented Dec 14, 2021 at 9:30
  • 1
    $\begingroup$ Your RSS and PRESS are still defined to be equal, which I don’t think you mean. // Isn’t there a trick to calculate PRESS without having to fit a new model each time a point is omitted? $\endgroup$
    – Dave
    Commented Dec 14, 2021 at 9:58
  • $\begingroup$ @Dave - there is a very fast way of updating $(X^TX)^{-1}$ if you modify a small number of elements in $X$, or more generally updating $A^{-1}$ if you modify a small number of elements in $A$ (Sherman-Morrison-Woodbury), Is that what you're thinking of? $\endgroup$
    – jbowman
    Commented Dec 15, 2021 at 0:49
  • $\begingroup$ @jbowman I mean what Richard Hardy mentions in his comment to me here. $\endgroup$
    – Dave
    Commented Dec 21, 2021 at 22:48

1 Answer 1


Consider what $R^2$ intends to do.

$$ R^2 = 1 - \dfrac{\overset{N}{\underset{i=1}{\sum}}\big(y_i -\hat y_i\big)^2}{\overset{N}{\underset{i=1}{\sum}}\big(y_i -\bar y\big)^2} $$

Focus on the fraction: $\dfrac{\overset{N}{\underset{i=1}{\sum}}\big(y_i -\hat y_i\big)^2}{\overset{N}{\underset{i=1}{\sum}}\big(y_i -\bar y\big)^2} $. That is the ratio of residual sum of square (RSS) values for two models.

In the numerator, we have the RSS for the model we are considering. In the denominator, we have the RSS for a naïve model that always predicts the mean of $y$; that is, $\hat y_i = \bar y$ for all $i$.

When you go to an out-of-sample test, to fit in the spirit of comparing to a naïve model that always guesses the mean of $y$, compare the out-of-sample $y$-values to the mean of the in-sample $y$. After all, it wouldn't really be a model trained on in-sample data if it predicted the out-of-sample mean.

When we get to $Q^2$, we apply a similar idea. PRESS is equal to the sum of the squared residuals for the holdout samples of leave-one-out cross validation (LOOCV). To fit with the spirit of $R^2$ of comparing to a naïve model that always guesses the in-sample mean, we calculate the squared differences between out-of-sample points and their predictions from naïve models that always guess the in-sample mean. Applied to LOOCV, this means that we find the squared residual coming from comparing $y_1$ to the mean of all other $y$ values, which I will denote by $\bar y_{-1}$. Then we do it again for $y_2$ and $\bar y_{-2}$. Then again for $y_3$ and $\bar y_{-3}$...

This is not equal to the $TSS$ value from the usual $R^2$.

y <- c(3, 5, 8)
tss <- sum((y - mean(y))^2)
t_q <- 0
for (i in 1:length(y)){
  loocv_value <- y[i]
  mean_of_the_rest <- mean(y[-i])
  my_value <- (loocv_value - mean_of_the_rest)^2
  t_q <- t_q + my_value
tss # 12.667
t_q # 28.5

Consequently, for $Q^2$ to preserve the idea of comparing our model to the naïve model that always predicts the in-sample mean, the denominator in $Q^2$ should not be the same as the $TSS=\overset{N}{\underset{i=1}{\sum}}\big(y_i -\bar y\big)^2$ denominator in $R^2$.

From the question you linked, it looks like they are going ahead and using the usual $TSS$ denominator anyway. The only value that I see to this is that using it for model selection is, in some sense, equivalent to using the full PRESS statistic, much like how using MSE is equivalent to using $R^2$ (lowest MSE for a model on a data set corresponds to highest $R^2$), except that PRESS penalizes excessive model complexity. Then we divide through by some kind of normalizing constant.

I do not consider that to be in the spirit of an $R^2$ that is tweaked for the LOOCV setting. However, if the goal is to find a model that minimizes out-of-sample square loss and the approach to choosing such a model involves LOOCV, the $Q^2$ calculation that uses the usual TSS is fine and equivalent to using the LOOCV sum of squared residuals.

(I haven't done the calculation or tried to come up with a counterexample in R, but it might turn out that model selection based on $1-\dfrac{PRESS}{TSS}$ and $1-\dfrac{PRESS}{\overset{N}{\underset{i=1}{\sum}}\big(y_i - \bar y_{-i}\big)^2}$ are equivalent. I would be quite delighted to see this be the case, and it would not surprise me, either.)


Define $DSS:=\overset{N}{\underset{i=1}{\sum}}\big(y_i - \bar y_{-i}\big)^2$ ("Dave's Sum of Squares").

Yes, the two turn out to be equivalent in the sense that a lower PRESS value will always result in a a higher $1-\frac{PRESS}{TSS}$ and also a higher $1-\frac{PRESS}{DSS}$. Consequently, any ranking of model quality based on $1-\frac{PRESS}{TSS}$ will be the same as the ranking of model quality based on $1-\frac{PRESS}{DSS}$.

If $P_1$ and $P_2$ are the $PRESS$ values from competing models of the same data, then:

$$ P_1 < P_2 \iff 1-\dfrac{P_1}{TSS} > 1-\dfrac{P_2}{TSS} \iff 1-\dfrac{P_1}{DSS} > 1-\dfrac{P_2}{DSS}$$$$ \text{AND}$$$$ P_1 > P_2 \iff 1-\dfrac{P_1}{TSS} < 1-\dfrac{P_2}{TSS} \iff 1-\dfrac{P_1}{DSS} < 1-\dfrac{P_2}{DSS} $$

In fact, establishing this involves no statistics or machine learning, just algebra. We would agree that showing the following is fair for middle school or high school algebra, right? (Assume that $D,T,X,Y>0$, since they will be in our machine learning problems.)

$$ X < Y \iff 1-\dfrac{X}{T} > 1-\dfrac{Y}{T} \iff 1-\dfrac{X}{D} > 1-\dfrac{Y}{D}$$$$ \text{AND}$$$$ X > Y \iff 1-\dfrac{X}{T} < 1-\dfrac{Y}{T} \iff 1-\dfrac{X}{D} < 1-\dfrac{Y}{D} $$

$1-\dfrac{k}{T}$ and $1-\dfrac{k}{D}$ are just monotonic transformations of $k$, so the order is preserved (yes, reversed, but the goal is to have low PRESS and high $Q^2$, so we want the order reversed).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.