Suppose I have a classification problem about whether or not a person has blue eyes. There is a training set with 87,000 observations and a test set with 22,000 observations. Now suppose after fitting a model and predicting on the test set I get this confusion matrix:

enter image description here

So I get a false positive rate of 25% and a false negative of 2%.
Now I run the model on a much bigger data set, 5 million observations.
If my goal is to estimate the total number on persons with blue eyes in this population, can I correct this number by the false positive and false negative rates?
In this example that would mean reducing the total number I got by 2% and then adding 25%.

My question is really how much can I rely on the results from the test set and is it acceptable to correct the total number by these results.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.