Data:
My training set consists of ~450k obs and 26 variables, out of which 1 is an ordinal factor (order_month, 12 levels) and the rest is numerical. Moreover, some of my predictors are highly correlated (Pearson's > 0.5), which is expected and sometimes intentional.
I trained a good RF model (ROC 0.96, ~90% specificity & ~90% sensitivity when tested on unseen data) and I derived Variable importance for it using
varImp()
.
Problem:
Now, I know that having factorial and / or correlated predictors could result in biased variable selection in RF models, to which conditional inference trees (CIT) should be a remedy.
I tried to compare results of
randomForest::varImp()
withparty::varimp(myforest, conditional=TRUE)
however I get an error in the latter:
Error in model.matrix.default(as.formula(f), data = blocks) :
term 1 would require 9e+12 columns
,
suggesting that I have either too many variables or variables with too many levels.
- On top of that it takes over 24hrs to run
party::cforest()
and runningpredict()
on CIT model crashes my computer altogether. Given the above, I have a couple of
Questions:
1) how likely is it that I may completely misclassify the top VarImps in RF model. Is running CIT here absolutely necessary?
2) what are alternative ways of obtaining conditional variable importance if not through party::varimp()
?
Big thanks for your suggestions! Kasia