I'm trying to model an ordinal customer satisfaction score with a mix of categorical and continuous predictors, and am just looking for some advice/critiques of my plan of attack by some of those more knowledgeable than myself. The DV was originally on a 1-10 scale, but has been binned (essentially just due to tradition within the company), the balance of classes is roughly 48/30/22 (%), there are 4000 observations with 11 predictors. The goal of this is to identify areas (predictors) which are most relevant to the customer satisfaction score and then to spend efforts focusing on improving those areas within the company.
My idea is to present some EDA in the form of graphs and the like of variables of interest, run an ordinal logistic regression to attempt to get a rough estimate of effect sizes/significances (possibly using adjacent categories), and finally to use something like
cforest or perhaps both to get an estimate for the relative ordering variable importances for the best predictive model.
Specifically my questions/concerns are:
Setting seeds: I've read I should be varying the seed when running RF-based models and checking the variable importance in each case. Does it make sense to average the relative importances of these different cases? i.e. I do this twice and predictor
x1ends up ranked in the top place out of 3, but then ends up in the last place on the next run, so gets averaged to rank 2.
The party package has a
conditionaloption within it's
varimpfunction which, as I understand it in part, attempts to quell the influence of spurious correlations on the variable importance rankings. This seems to be useful, though time intensive, but can anyone suggest perhaps why I would not want to use it?
Apologies for long winded-ness, but this is my first project for the company (which doesn't have anyone more knowledgeable than myself of these types of analyses) in my first role outside of school, so am trying to handle it as thoroughly as possible.
Thank you much.