I'm currently using random forest to determine how a set of temporal predictors, lagged and aggregated at many different timescales, influences a binary disease outcome. My main goal is to understand the relationship between the predictors and the outcome (which I expect to be highly non-linear) and to determine which temporal lags/aggregations perform the best. In other words, yes I'm interested in building a model that performs well in a predictive sense, but I'm even more interested in understanding the underlying relationships.

I've been using a great R package called forestFloor, which employs feature contributions (an improved version of a partial dependence plot) to visualize and quantify the marginal effects of each predictor. However, when I tune mtry to maximize predictive performance, the most important variables include the same predictor lagged at a few different timescales, because they are correlated. Also, the forestFloor feature contributions of each these lagged predictors is relatively low because the total contribution of the variable is being split among a few correlated lags.

I'm wondering if would be appropriate in this case to force the trees to be more highly correlated by setting mtry to be the total number of predictors in the model, such that the very best lagged predictor (or a few of the best) contributes to splits more often, gets a higher importance value, and has higher feature contributions. I'm also open to other strategies that would allow me to hone in on the very best predictors.

  • $\begingroup$ The boruta algorithm is a principled to analyze feature importance to select all relevant features. It's not terribly sensitive to mtry. $\endgroup$ – Sycorax Nov 27 '18 at 2:52
  • $\begingroup$ thanks Sycorax, I've used boruta as a pre-processing step, but I doesn't seem to remove correlated predictors (e.g. same predictor variable at different lags) that are all pretty good, even if one is substantially better $\endgroup$ – Nick_89 Nov 27 '18 at 20:45
  • $\begingroup$ That’s because Boruta isn’t designed to solve that problem. Boruta selects all relevant features, not the minimal optimal set of features. $\endgroup$ – Sycorax Nov 27 '18 at 21:34
  • $\begingroup$ yup, that's why I'm curious if maximizing mtry and thus correlating the trees might be a good way to get at this issue $\endgroup$ – Nick_89 Nov 27 '18 at 22:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.