What is statistical meaning of testing of machine learning model on new data? Any machine learning model, normally, is applied on new data for testing. If the accuracy of the model is below a certain threshold, the model is rejected. But why? This is just a random sample, the result may not represent the performance of the model on the whole population. What is the statistical meaning of this operation?
 A: Ah - fresh to statistics. If you have a test set of data from say $10.000.000$ customers, their postal addresses, their annual purchase in your webshop and a few other variables - then the randomness of such a large sample is neglectable. If, on the other hand, you have $50$ customers in your test set, then the randomness so much more is present.
The confidence intervals of your model predictions become narrower with more test data. These test data must come from the same underlying distribution as the training data used to build your model. Otherwise, the test set will be biased in relation to your training set.
The minimal threshold for an acceptable model accuracy, that depends on the application domain.
A: I think, the reasoning is like this. If training set and/or test set has distribution randomly different from the distribution of general population, then, most likely, distributions on training and test data will be different from each other. In this case, the performance of the model on the test data will be poor. If the performance of the model on the test data is good, it signals that distributions of training and tests set are similar (as far as the model is concerned). And this, in turn, would imply that they both are close to the distribution on the general population, and the test results can be used to predict performance on the general population.
I doubt somebody proved it.
