1
$\begingroup$

I am building a model to predict the remaining life of a patient.

My data has 20 features and it is in the format of a life table where each observation represents a patient $j$ on time $t$ time until the time he died.

So, for each patient, I have a group of observations correlated between them.

The random forest model I built is doing a good job on training and also on test set, however, it concerns me a little bit those correlations. It would me concerns a lot if I was modeling by a GLM, however, I am not quite sure if it would be a problem by random forest.

$\endgroup$
2
  • $\begingroup$ What exactly is your concern? $\endgroup$ Nov 18, 2020 at 12:27
  • $\begingroup$ regarded with the assumptions of the model, does random forest assume independent observations? and if so, how does it impact on the estimation. $\endgroup$
    – motipai
    Nov 18, 2020 at 12:29

1 Answer 1

1
$\begingroup$

RF's don't really have any assumptions. Multicollinearity will affect the variable importances, if this is of interest to you, although this is in part dealt with by feature bagging. What may be of interest to you is the bootstrapping procedure.

By default, RF perform a vanilla bootstrap to generate resamples. This is a problem for time series data, because the autocorrelation in the data is not preserved or, as in your case, a sort of missing random effect for each individual to model the time dependence.

Some options: 1) modify the bootstrapping procedure to perform a different kind of bootstrap (good luck) suited for time series/dependent data (such as block bootstrap), 2) normalize the time component, so that it is comparable between the patients, although not the same as a random effect in a linear model.

$\endgroup$
4
  • $\begingroup$ RF's have many assumptions, but they are subtle, and in many common cases don't impact the work. $\endgroup$ Nov 18, 2020 at 13:44
  • 1
    $\begingroup$ @EngrStudent Could you expand on that comment? What assumptions do RFs have, and why do they not often impact the work? $\endgroup$
    – Sycorax
    Nov 18, 2020 at 13:54
  • 1
    $\begingroup$ @Sycorax - it is dangerous when a 62k rank says "expand", ;) The CART splits are perpendicular to the axes, often using Gini impurity. This implicitly assumes a balance between feature alignment with the axes or high-enough sample sizes to track the hyper-diagonals as indicated by Gini. The mode or mean aggregation assume the mode or mean are the right summary aggregators. Same for leaf-tip aggregation using mean or mode. Some tools handle the many-level bias of RF, but others don't account for it. Bootstrap columns vs. number of trees vs. max depth vs. samples per tip interact. $\endgroup$ Nov 18, 2020 at 16:16
  • $\begingroup$ @EngrStudent I think you've made a helpful addition -- these are good things to keep in mind when using RF. No danger here! $\endgroup$
    – Sycorax
    Nov 18, 2020 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.