Oversampling with categorical variables

Question

I would like to perform a combination of oversampling and undersampling in order to balance my dataset with roughly 4000 customers divided into two groups, where one of the groups have a proportion of roughly 15%.

I've looked into SMOTE (http://www.inside-r.org/packages/cran/DMwR/docs/SMOTE) and ROSE (http://cran.r-project.org/web/packages/ROSE/ROSE.pdf), but both of these create new synthetic samples using existing observations and e.g. kNN.

However, as many of the attributes associated with the customers are categorical I don't think this is the right way to go. For instance, a lot of my variables such as Region_A and Region_B are mutually exclusive, but using kNN the new observations may be placed in both Region_A and Region_B. Do you agree that this is an issue?

In that case - how do one perform oversampling in R by simply duplicating existing observations? Or is this the wrong way to do it?

Answer 1

ROSE and SMOTE are designed to handle categorical variables, so, unless your categorical variables are expressed in a binary format, you shouldn't normally have to worry about synthetic observations being assigned mutually exclusive categorical features. If they are, you can always restructure them as factors.

In your two-region example, you would create a new region variable with two levels, "A" and "B". Your records would take the appropriate values by referencing your original columns.

Now, if you are in a situation where your new synthetic observations could generate conflicting categories because they are spread across multiple, otherwise unrelated variables (e.g. syntheticObservation.isPig = 1 and syntheticObservation.hasWings = 1), you could always perform some additional data munging before doing your model estimation in order to clean such aberrations.

Also, since you do have about 600 event observations in your dataset, maybe consider the potential benefits of using synthetic observations derived through undersampling the majority class?