Let's say we've some training data points (not a huge number), collectively referred to as $X$, and we're trying to fit a curve (a polynomial) to these points. Later, we check training and test set errors.
We know for a fact that higher degree polynomials are bound to overfit the data, and relatively lower degree polynomials might give smaller errors on the test set.
Mathematically, though, a (lower degree) polynomial is just a subset of any polynomial with comparatively greater degree. So, it should be possible (is there some way, rather?) for a higher degree polynomial that overfits the training set, and gives wild results on the test set, to perform at least as good as lower degree polynomial that does not overfit (since that's just a subset)
So, how can the loss term be adjusted to incorporate the test set? Ideally, we're not supposed to see the test set until we're done training, so I'm in bit of a confusion here!