Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 1 down vote favorite

I have a data set with ~ 7 million rows, of which ~ 100k are positives. I'm looking to shrink the data by keeping all the positives and then randomly sampling several hundred thousand negative examples to round out the data set.

I'm uncertain what the guidelines are though. Are there any rules of thumb for what % needs to be positive / negative?

Any suggestions would be appreciated.

share|improve this question
So what are you trying to achieve? What are your analysis goals? And why do you think you need to subsample? What exactly the problems are with the full data set? – StasK Sep 4 '12 at 23:34
@Stask: I'm trying to predict whether someone will respond to a mailing (binary classification). My main goal is to work with a data set that is < 1 GB instead of one that is 10 GB. – screechOwl Sep 4 '12 at 23:43
You don't always have to work with the 10Gb set, you could use methods like a decision tree or random forests to generate the classifier and then work with that; the classifier won't require that much information compared to the information needed to train it. – casperOne Sep 5 '12 at 1:49

1 Answer

up vote 1 down vote accepted

If you keep all the positives from your data set you may find that you have skewed your results.

A priori the probability of a positive in your data is about 1 in a hundred. If you down sample so you have 100K +ve and 100K -ve the a priori +ve probability is now 1 in 2.

Unless there is a large separation with little overlap between the two classes you will most likely create a strongly biased classifier.

As a first step create a smaller stratified sub sample and see what performance you can achieve with that.

Then you can investigate how your classifier behaves if you increase the percentage of +ves in the training set and use a test set with a much higher percentage of -ves. This should give you some idea of the sensitivity of your methods to class balance.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.