I used a RF model to predict a rare event. After testing using multiple iterations, I settled on a model which provided decent results. I tested this model on a validation data I had kept aside before training the model. The result is similar here also. Now i extracted the votes(ie the probabilities)from the RF model for each training observation and also the probabilities from test & validation runs. Now I have a data frame of all my observation plus a column of the probabilities predicted by the random forest model. I want to preform clustering now on this data to understand the predictor values that are a part of my high probability conversion cluster. Any advice on what tools would be best to explore these clusters.

  • $\begingroup$ Why cluster at all? Why not simply create deciles or ventiles based on the predictions, rank the results from high to low average probability, then explore the highest ranking groups? It's much easier. Also, you might want to consider adding a column of residuals for analysis. $\endgroup$ Mar 19, 2017 at 15:46
  • $\begingroup$ @DJohnson thanks for the reply.. i wanted to do that but I also wanted to know the combinations.. for eg when Variable 1 has value <3 and Variable 2 is >200..etc they fall in a group of High Probability... $\endgroup$ Mar 19, 2017 at 16:24
  • $\begingroup$ Sure. So, based on the grouping, average the variables within each group. Then index the group averages relative to the overall average for each variable creating a kind of IQ score or "t-test" proxy where around 100 is average or "normal" behavior for a variable, 120 and above is a "highly likely" characteristic and 80 and below is a "highly unlikely" characteristic. These are simple, directional heuristics that tell a story. $\endgroup$ Mar 19, 2017 at 16:49
  • $\begingroup$ @DJohnson Thanks, I have all categorical data 24 variables including 2 continuous variables which I binned to make them categorical. Do you have any suggestions that can help here.. $\endgroup$ Mar 20, 2017 at 13:58
  • $\begingroup$ What's your objective with this analysis? To identify a subset of variables that are most important or predictive of probability? To identify specific levels within variables that are most associated with predicted probability? Also, RFs may not be the best approach for analyzing and predicting sparse data. Have you explored any other methods? $\endgroup$ Mar 20, 2017 at 14:23


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.