2
$\begingroup$

I am trying to compare models using K-Fold-CV using the regsubsets function in R.

By default, it states that the ideal model is determined by the $RSS$.

I wished to change this parameter such that ideal model selection was performed on the basis of the $R^2$ value. However, I noticed that:

$R^2 = 1 - (RSS/TSS)$, and $TSS = \sum \big(y_i-\bar y\big)^2$. And therefore, is ideal model selection based on $RSS$ essentially equivalent to ideal model selection by $R^2$, given that the $TSS$ value is constant between test data-sets?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

They are equivalent for the exact reason that you mention: one is a monotonic transformation of the other. Just keep in mind that we want to maximize $R^2$, while we want to minimize the residual sum of squares.

Since we typically want to find the best model for a set of data, we typically have a constant $TSS$.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Note that $R^2$ can be made quite large (close to $1$) by adding many parameters that fit the noise but would not generalize. Residual sum of squares suffers from the analogous issue of being able to be made quite small (close to $0$) by using many parameters that fit to the noise in a way that would not generalize. The two approaches are, however, equivalent in the way described in the original question and in my answer. $\endgroup$
    – Dave
    Commented Dec 1, 2021 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.