Context: Have been trying to create a prediction model for a 1% outcome variable using Random Forest Machine Learning for a large health survey (entirely multi-level categorical data, yes/no outcome, ~230,000 observations initially). Stupidly high amount of missing values, poor data collection and the need to infer between variable. Multiple imputation used, CCA used (as a separate base), k-fold CV, train/test split, the lot. Whilst under-sampling improved things marginally, over-sampling was pretty much consistent with no sampling at all (~140,000 observations now, with perfect sensitivity and no specificity reached - i.e., unhelpful). Whilst Random forest can handle this type of data well, I understand it has its limitations (especially with class imbalance and biasness towards certain predictors).
I've read a bit of ML research, and I don't claim to understand all of it, but there's sometimes a pretty solid paper on improper methods for ML. Presently, I understand multiple logistic regression SHOULD be able to handle the class imbalance - but in my case, I still get nearly perfective sensitivity and 0 specificity.
Does this then justify the use of a balancing method in that the imbalance is now a problem, or am I missing something? Have just been using the Caret train package for most training needs.