1
$\begingroup$

Given a Bernoulli Process, should my training set have a number of "1" examples in proportion to the process?

For example, a Bernoulli Process is "1" 10% of the time and "0" otherwise. In a training set of 1,000,000, should I use 10,000 "1" examples and 90,000 "0" examples?

Back-story: I have a large training set of 100 billion rows. I have about ~200,000 "1" cases, about .2% of the time. Training will take forever so I want to do a subset of this data. Taking a straight sequential chunk of this data, I'm afraid I won't have any "1" cases contained in the subset. But now I'm wondering if the way I sample this training data would affect my classifier.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.