Introduction
I have a small mixed dataset consisting of continuous and categorical independent variables with a dichotomous dependant variable. I'm running various algorithms (neural networks, random forests and logistic regression) to find the best model to generalise on unseen data. To do this i have performed a nested cross validation so that I can train and optimise hyper parameter and feature selection (on the inner loop) without being too biased on the model performance. I have conducted a 10 fold cross validation in the inner loop and outer loop.
My Question
My dataset has a class imbalance. To overcome this problem I have oversampled with replacement the minority class so that the majority is now at 80% and the minority at 20%. I know this still has imbalance, however it is now more appropriate to what can be found in the field and better than the original imbalance. My question is will this cause bias in the model performance because I used a oversampling technique with replacement?