I have a predictor that exhibits "bimodal" behaviour. How can I engineer this feature to improve performance within a Decision Tree?
For an intuitive example, consider how a binary flag of "moves house in next 6 months?" may correlate with a continuous variable of "change in income over last month". This may yield a distribution like the following:
- those that get larger increase in income (e.g., get promoted) are more likely to move house (e.g., up-size) than those whose income remains unchanged
- those that get larger decreases in income (e.g., made redundant) are also more likely to move house (e.g., move back in with parents)
My trouble is that when using a Decision Tree (non-negotiable), cuts are applied on nodes to find the best value at which to split. However, due to the bimodal nature of the feature, the cuts cannot successfully split the data well enough.
Question: how can I engineer this feature to improve performance within a Decision Tree?
I have considered:
- adding an additional feature that states whether the change is positive or negative
- converting the data into categories (bins), which can later be one-hot-encoded