# oversampling in nested cross validation

Introduction

I have a small mixed dataset consisting of continuous and categorical independent variables with a dichotomous dependant variable. I'm running various algorithms (neural networks, random forests and logistic regression) to find the best model to generalise on unseen data. To do this i have performed a nested cross validation so that I can train and optimise hyper parameter and feature selection (on the inner loop) without being too biased on the model performance. I have conducted a 10 fold cross validation in the inner loop and outer loop.

My Question

My dataset has a class imbalance. To overcome this problem I have oversampled with replacement the minority class so that the majority is now at 80% and the minority at 20%. I know this still has imbalance, however it is now more appropriate to what can be found in the field and better than the original imbalance. My question is will this cause bias in the model performance because I used a oversampling technique with replacement?

• Why do you think the class imbalance is a problem, and what problem is the oversampling addressing? Oct 18, 2017 at 16:57
• Hi Matthew, Class imbalance is a problem because the algorithms im working with tend to favour the majority class and disregard the minority. Oversampling will hopefully prevent this bias in the model Oct 18, 2017 at 17:24
• Can you say more about the algorithms you are working with? Oct 18, 2017 at 18:33
• Hi matthew, im working with the algorithms i mentioned in the original question Oct 23, 2017 at 21:26

Oversampling will clearly introduce bias in your model because $P(class|data) = \frac{P(data|class)P(class)}{P(data)},$ and you're artificially messing with $P(class)$.