My training set consists of ~450k obs and 26 variables, out of which 1 is an ordinal factor (order_month, 12 levels) and the rest is numerical. Moreover, some of my predictors are highly correlated (Pearson's > 0.5), which is expected and sometimes intentional.
I trained a good RF model (ROC 0.96, ~90% specificity & ~90% sensitivity when tested on unseen data) and I derived Variable importance for it using
Now, I know that having factorial and / or correlated predictors could result in biased variable selection in RF models, to which conditional inference trees (CIT) should be a remedy.
I tried to compare results of
party::varimp(myforest, conditional=TRUE)however I get an error in the latter:
Error in model.matrix.default(as.formula(f), data = blocks) :
term 1 would require 9e+12 columns,
suggesting that I have either too many variables or variables with too many levels.
- On top of that it takes over 24hrs to run
predict()on CIT model crashes my computer altogether. Given the above, I have a couple of
1) how likely is it that I may completely misclassify the top VarImps in RF model. Is running CIT here absolutely necessary?
2) what are alternative ways of obtaining conditional variable importance if not through
Big thanks for your suggestions! Kasia