1
$\begingroup$

In a hypothetical setup where train and test set features have identical distributions, is the correct to say that one can not overfit.

$\endgroup$
0
$\begingroup$

Not really. You overfit to observed data, not distribution of the training data. Simple example is where your data is a limited number of points and you fit large degree polynomial regression to it. The regression line could fit the points exactly, while this could have not much to do with any sample that you'd observe when sampling from your test set distribution.

Saying it differently, to test for overfitting you usually split your data to training and test sets, so they both come from same distribution, and in many cases this is still enough to observe overfitting.

Moreover, if your training and test data come form different distributions, then something is wrong with the data you use. If they come from different distributions, you shouldn't use this model to make predictions on the test set, since the model was simply not trained to make predictions on such data, so you have no guarantees it will work.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.