# Are RSS and R^2 related to training error only?

While reading An Introduction to Statistical Learning, I stumbled across the following (p. 210):

[...] the model containing all of the predictors will always have the smallest $RSS$ and the largest $R^2$ , since these quantities are related to the training error.

The text talks about feature (input) selection in regressive models. The reference is the following (p. 205):

The problem is that a low $RSS$ or a high $R^2$ indicates a model with a low training error, whereas we wish to choose a model that has a low test error.

I don't understand why the authors relate those error measures to the training dataset, since we can calculate $RSS$ or $R^2$ on a test dataset (given that I have enough data) too.

Is there any reason for this distinction?

• it would be helpful if you gave a larger context, but yes they do mean training RSS, and there is nothing to stop you applying RSS or R^2 on test data set ( and they discuss various approaches to estimating test performance later in the book) – seanv507 Jul 3 '16 at 18:41
• @seanv507 Well, the context is selection of inputs (subset feature/variable selection) in linear model selection. Still, the authors say that >the same concepts apply to other methods. In any case, I was just wondering if there's theory or reason that I don't know, because they relate $RSS$ + $R^2$ to training only. In my understanding, the main thing is to validate and test (using any measure of error) with other data, if available, and that was it. – Luis Jul 4 '16 at 10:47
• The authors use RSS often to mean training RSS (as in your quote), but do also talk about test RSS eg Q.4 on page 120-21) – seanv507 Jul 4 '16 at 11:39