I have a highly imbalanced data set: a total of 13000 patients, 160 having condition A, and various other features which could be predictors.
In order to balance the data I did two things:
1) Matching on some of the variables to neutralize the variance (the ones balanced on were not found as predictors in stepwise regression)
2) Using the SMOTE algorithm to create more synthetic data from the rare condition on the one hand, and decreasing the number of samples from the majority class, such that eventually my minority class was 1/3 of the entire data.
SMOTE gave a much better AUC after stepwise regression. In addition, matching on more and more variables in step 1) above gave a lower and lower AUC.
What is more correct to do?