My data has 13,000 rows, 7% belong to the minority class. I used SMOTE (Synthetic Minority Oversampling TEchnique) for class balancing such that I raised the ratio of minority class to 42 % and number of rows became 12,655. Now to fit a logistic regression I need to divide the sample for cross validation and testing. I tried two approaches:

  1. train my data on sample obtained after SMOTE and tested on the
    original sample having 13,000 rows,
  2. divide the sample obtained after SMOTE into train and test and do the fitting and testing on this data set only.

With the first approach my results might get skewed so which approach should I adopt and why?

  • $\begingroup$ I do not think resampling is the best idea here. The logistic regression model will already be skewed and twisted towards the majority class, i.e. it favors it with an implicit prior. First question would be: do you want that behaviour? If yes, that would imply the underlying distribution of your data also is heavily in favor of the majority class just like with your sample. $\endgroup$ – pAt84 Feb 15 '16 at 10:43

Reampling should be applied to training set only, and then you should test on a subset having the same class distribution as the population. If you oversample the minority class in the test set, you may get a higher hit rate i.e. better performances than in the real case, where positive instances are rare.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.