While reading An Introduction to Statistical Learning, I stumbled across the following (p. 210):
[...] the model containing all of the predictors will always have the smallest $RSS$ and the largest $R^2$ , since these quantities are related to the training error.
The text talks about feature (input) selection in regressive models. The reference is the following (p. 205):
The problem is that a low $RSS$ or a high $R^2$ indicates a model with a low training error, whereas we wish to choose a model that has a low test error.
I don't understand why the authors relate those error measures to the training dataset, since we can calculate $RSS$ or $R^2$ on a test dataset (given that I have enough data) too.
Is there any reason for this distinction?