Given a Bernoulli Process, should my training set have a number of "1" examples in proportion to the process?

For example, a Bernoulli Process is "1" 10% of the time and "0" otherwise. In a training set of 1,000,000, should I use 10,000 "1" examples and 90,000 "0" examples?

Back-story: I have a large training set of 100 billion rows. I have about ~200,000 "1" cases, about .2% of the time. Training will take forever so I want to do a subset of this data. Taking a straight sequential chunk of this data, I'm afraid I won't have any "1" cases contained in the subset. But now I'm wondering if the way I sample this training data would affect my classifier.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.