3
$\begingroup$

If I'm building a decision tree model, what is the best way to incorporate features that are always 0 given the value of another feature?

For example, imagine I'm predicting whether or not someone has high blood pressure, and two of the many features I have are gender and whether or not the person has uterine cancer. Clearly if gender is male, then uterine cancer has to be 0 because men cannot get uterine cancer. I'm fairly positive this can be done with interaction terms, but decision trees cannot utilize them.

$\endgroup$

2 Answers 2

1
$\begingroup$

Decision trees (e.g. CART) should not be affected.

In the male branch of the tree, the cancer variable would be always 0. Therefore, it has 0 statistical power. And the tree will not learn any splits based on the cancer variable.

$\endgroup$
3
  • $\begingroup$ No splits in the male branch, or no splits in both the male and the female branch? $\endgroup$ Commented Jul 19, 2016 at 19:46
  • $\begingroup$ Hey @BrandonSherman, just the male branch. The cancer variable has predictive power in the female branch, so there will be a split on the cancer variable if your max depth allows it. $\endgroup$ Commented Jul 19, 2016 at 19:56
  • $\begingroup$ Ah okay. So it does the split automatically, so there's no need to manually create an interaction. $\endgroup$ Commented Jul 19, 2016 at 20:12
1
$\begingroup$

Technically speaking, your Gender and Cancer factors are neither crossed nor nested. If they were nested, then you could just specify the interaction term, but I am not sure whether it will work correctly in this case.

Therefore, I think the best way to avoid confusion is to introduce a new factor Gender_Cancer with 3 levels: "M_No", "F_Yes", "F_No". Then in order to test the effect of gender, you specify the contrast as "M_No vs F_No". To test the effect of Cancer, the contrast is: "F_Yes vs F_No".

$\endgroup$
1
  • $\begingroup$ (+1) This is a good answer because it takes advantage of an intrinsic ability of decision trees to study interactions. Besides, it's one less feature to try splits, so might speed up calculations a bit. Also, it's useful because splits on sex variables might not be near the root node. $\endgroup$
    – Firebug
    Commented Jul 19, 2016 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.