2
$\begingroup$

I am working on a project to determine the variables that better predict the binary outcome. I am using conditional random forest and permimp::permimp for determining the importance of variables for my subgroup analysis.

Now, I want to determine the cut-offs for the continuous variables which could help to predict the outcome. Could you please suggest the method to do so? there are so many methods so I got lost

Improve this question
$\endgroup$
15
  • 1
    $\begingroup$ Wouldn’t there be a different splitting value (cutoff) in every tree in the random forest? $\endgroup$
    – Dave
    Commented Jan 19 at 10:35
  • $\begingroup$ @Dave something like the average/median cut-off used within forest? $\endgroup$
    – Kate
    Commented Jan 19 at 10:46
  • 2
    $\begingroup$ The fact that this only has the r tag makes it seem that the question is just looking for some code, which remains off-topic here (not an inherently bad question, just outside the purview of Cross Validated). Perhaps this should be reopened to allow for a statistical response, however, perhaps even one that disputes the utility of the hard cutoffs the OP seems to seek. $\endgroup$
    – Dave
    Commented Jan 20 at 10:43
  • 1
    $\begingroup$ I'm not very familiar with tags (nor with the topic of the question) but I would replace the r tag with classification tag. $\endgroup$
    – Pere
    Commented Jan 20 at 13:58
  • 4
    $\begingroup$ As discussed extensively on this site, cutoffs are bad ideas. They don’t exist in nature because discontinuities don’t exist in nature unless X=time. Since they don’t exist, every analyst will find a different cutoff. $\endgroup$
    – Frank Harrell
    Commented Jan 23 at 12:36

1 Answer 1

Reset to default
5
$\begingroup$

Don't do this, as it particularly doesn't make sense for a random forest model. In addition to the many reasons that categorizing a continuous predictor is a bad idea, it undercuts a potential strength of a random forest: its ability to find unsuspected interactions among predictors in a model.

With a random forest, the association of one predictor with outcome can thus depend on the values of other predictors. Any cutoffs you might choose would thus necessarily depend on the values of other predictors. Why not just use the full model to make predictions as needed?

In response to comment

The above applies to post-modeling cutoffs, as constructing the random forest already involves making multiple cutoffs of each continuous-predictor value. At lower branches of each of the many trees that are built, the choice of cutoff for one predictor will depend on the values chosen for all the predictors used at higher levels of that tree. After the model is built there will be no one cutoff for any continuous predictor that is independent of the values of the other predictors.

This is even the case for simple standard regression models that contain simple interaction terms. For example, if the effect of one continuous variable (e.g., age) depends on another (e.g., hemoglobin A1C) in an interaction term, then you would have to change the cutoff for age depending on the value of hemoglobin A1C.

A clinician is presumably interested in the probability of developing the disease. That depends on combinations of values of all the predictors. You can make probability predictions from a properly constructed random forest, based on each new patient's values.

Even if a clinician and patient might then choose some probability cutoff based on the relative costs of false-positive and false-negative assessments, that probability will depend on (usually unknowable, with a random forest) combinations of all the predictors that were identified during model construction.

Improve this answer
$\endgroup$
6
  • $\begingroup$ sorry for not being clear in the post - I pass continuous variable to fit random forest and to identify the important variables. Once important variables are identified, I would like to identify the cut-off for continuous variables since it is valuable for clinicians. So I am not applying the cutoff before fitting random forest $\endgroup$
    – Kate
    Commented Jan 25 at 7:37
  • $\begingroup$ @Kate I was talking about looking for cutoffs after the model is built. I've added a bit of extra explanation. A random forest isn't like a single decision tree, where the modeling might have found single cutoffs for each continuous predictor. $\endgroup$
    – EdM
    Commented Jan 25 at 9:17
  • $\begingroup$ To summarize and highlight @EdM's points: You are proposing identifying cutoff points as if what mattered were bivariate relationships. This is inconsistent with the use of a more sophisticated, multivariate method (random forest) which, like multiple regression, will go beyond such bivariate relationships to show cases that, by virtue of their predictor combinations, have high probabilities re: the outcome. $\endgroup$
    – rolando2
    Commented Jan 25 at 11:16
  • 1
    $\begingroup$ As @EdM stated, the use of cutoffs is a really, really bad idea. Though often used in clinical medicine, users of cutoffs on measurements don’t realize the the cutoff of a predictor must be a function of all the actual levels of the other predictors. This is demonstrated in hbiostat.org/bbr/info#categorizing-continuous-predictors $\endgroup$
    – Frank Harrell
    Commented Jan 25 at 14:30
  • 1
    $\begingroup$ @Kate I'm not an expert on random forests and don't use them much. I would suggest posting a separate question on this site about what to do in that scenario. In that question, please provide as much detail as possible about the nature of the outcome and the predictor variables, and the reasons why you might not have data on all the predictor variables. My understanding is that random forests can handle missing data fairly well, but you would be better off getting advice from someone who uses them routinely. $\endgroup$
    – EdM
    Commented Jan 25 at 15:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.