In a book I'm reading I've come across the following quote:

Accuracy on the test set is a good performance measure only when there is a relatively uniform distribution of target categories in the training set.

If the real world categories are distributed 10% A and 90% B, then should the training categories also be 10% A and 90% B, or should they be 50% A and 50% B?


marked as duplicate by Sycorax, mdewey, steffen, kjetil b halvorsen, whuber Jul 29 '18 at 17:08

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.