Random forest offers a simple way to measure the importance of each variable. But is there a way to measure the combined importance of two variables in the model?

For example, a model seeks to predict whether a student could be admitted by college. The independent variable is the examination score, IQ score, and height. Random forest model gives a feature importance for each of the three variables separately, but what if I’m interested in the combined importance of examination score and IQ score combined?

  • $\begingroup$ Before trying to answer that how do you know that your sample size is large enough for RF? RF requires perhaps 200 events per candidate feature to be reliable. RF is meant for very tall and very thin datasets (lots of observations, not too many candidate variables). RF is also notorious for resulting in poorly calibrated predictions. If your model's predicted probabilities are not well calibrated, the whole exercise is fruitless. $\endgroup$ Commented Oct 14, 2021 at 12:02
  • $\begingroup$ That's right. My data is considerably small. Maybe adaptive LASSO is a better choice? I am struggling to understand the difference in application between adaptive LASSO and random forest in feature selection. $\endgroup$ Commented Oct 14, 2021 at 14:14


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.