1
$\begingroup$

In context to supervised learning, I have been told that the training dataset and testing data set must be obtained from same distribution whichever it is. That is, for a given supervised learning algorithm, if training data is obtained from say, a normal distribution, then test dataset must also be obtained from normal distribution.

Why is this restriction?

How will the learning performance change when this restriction, if it really exists, is uplifted?

$\endgroup$
1
  • $\begingroup$ I'd say this is reasonable advice, not a strict requirement. $\endgroup$ Jan 11, 2017 at 9:54

2 Answers 2

5
$\begingroup$

Your training dataset is supposed to be an unbiased representation for your test dataset, otherwise you'd have a problem.

Your objective is to train a model that can generalize to a data set that you haven't seen, and that can only happen if what you use for training is approximately what your model will eventually encounter. No machine learning can compensate for bad and invalid inputs. You don't compare Apple and Orange.

$\endgroup$
0
$\begingroup$

If they come from different distributions, it is like trying your best to approximate an apple and then test your outcome with an orange.

In learning, the purpose is to find a "hypothesis" (say a function) which approximates a unique target function. This task is carried out based on a set of realisation of the target function. These realisations are usually divided into a train and a test set. At this point, hopefully, it is clear that both the train and test set are generated by that unique target function. Hence, they may not come from two different distributions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.