2
$\begingroup$

This question already has an answer here:

Say I have two training data of email titles with 10000 entries each. One of them have 2000 ham and 8000 spam, while the other have 5000 ham and 5000 spam. Will predicting any random email with the 2:8 training data tend to go for spam? Also is this necessarily a bad training data compared to the 1:1?

$\endgroup$

marked as duplicate by Matthew Drury, kjetil b halvorsen, Sycorax, mdewey, Peter Flom Oct 21 '17 at 12:48

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

migrated from stackoverflow.com Oct 19 '17 at 15:46

This question came from our site for professional and enthusiast programmers.

  • $\begingroup$ There is no programming question here.. this belongs on CrossValidated $\endgroup$ – juanpa.arrivillaga Oct 18 '17 at 22:20
  • $\begingroup$ Is there a way to move it there somehow? $\endgroup$ – Rizki Hadiaturrasyid Oct 19 '17 at 0:32
  • $\begingroup$ The answers you get here will be more subtle than the two answers already posted. Most good models (i.e. ones that predict the probability of class membership) do not suffer in any way from class ratio imbalance, the problem being more one of absolute class rarity. On the other hand, your decision procedure will have to deal with the class balance in some way, as there is always a balance between false negative and false positive rates when making go or no go decisions. $\endgroup$ – Matthew Drury Oct 19 '17 at 15:50
2
$\begingroup$

Quick answer, yes. Your model will be affected by the ratio of classes to predict. If 80% of your dataset is spam, depending on the model, it will be very difficult to differentiate the spam ones from the none spam ones.

Why don't you mix both data sets to have more balanced ratios, not necessarily 50/50 but it could be 40 / 60 something like that. Use the ratio that you think it's the best emulates reality.

$\endgroup$
  • $\begingroup$ How about news category? say there are 10000 entries divided evenly to 5 categories, and I'm comparing each one against the rest 5 times (so one entry can have more than one category). Since I'm comparing one against the rest it's technically 2000 vs 8000. Will it tend to output no category thus 'bad'? or is it actually good since in reality each categories are about evenly numbered? $\endgroup$ – Rizki Hadiaturrasyid Oct 18 '17 at 22:04
  • 1
    $\begingroup$ you mean you are comparing one category against the remaining four, right? I don't know exactly what you mean by comparing, if you are using for example a multinomial logistic regression you'll classify each example as only one of 5 categories. On the other hand, if you build 5 different logistic regressions (class or not class) for each one of the 5 different categories you have no "unbalanced" dataset problem. $\endgroup$ – Diego Aguado Oct 18 '17 at 22:18
1
$\begingroup$

Will predicting any random email with the 2:8 training data tend to go for spam? Also is this necessarily a bad training data compared to the 1:1?

Highly depends on the model you use (some allow the data to be unbalanced), but in general - yes. Try oversampling/downsampling for example.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.