0
$\begingroup$

The starting point is intuitive I guess: I want to do a decision tree type of algorithm, but some of the variables are continuous, e.g. numbers instead of quantities like ages.

I cannot afford the computation to directly treat them as discrete variable, I also don't want to lose much information through binning, and I don't know which binning method is the best.

Maybe regression will be a good choice. But I don't want to use categorical variables to build the tree first then do the regression at leaves. I think treating both kinds of variables equally will be better. In other words, I wish continuous variables could also show up in the early branch. Of course, I guess it will be hard for tree structure to achieve this.

So I'm wondering if there's such hierarchy structure which can use both continuous and categorical variables as keys?

| cite | improve this question | |
$\endgroup$
  • $\begingroup$ you edit it to clarify what you are trying to model? You refer to both regression and classification (both of which decision trees can be used for, incidentally), so the goal is a bit unclear. $\endgroup$ – mkt - Reinstate Monica Aug 28 '18 at 12:52
  • $\begingroup$ My bad. I believe I want to do regression, an interpretative regression. That's why I pick decision tree as start and use the average of the leave as final prediction. $\endgroup$ – G. Yu Aug 28 '18 at 13:29
  • $\begingroup$ Many machine learning algorithms have no problem with a mix of continuous and categorical variables. Decision trees are among them. There was at one point some difficulty with getting them to be treated equally (loosely speaking), but this was resolved a while ago, I believe. Random forests can certainly handle both types together. Does this answer your question? I'm still unsure based on your question. $\endgroup$ – mkt - Reinstate Monica Aug 28 '18 at 14:05
  • $\begingroup$ Thanks! Could you talk more about how to treat them equally or give me a link? I think that's exactly my question $\endgroup$ – G. Yu Aug 28 '18 at 14:35
  • $\begingroup$ Going by what you wrote, give a try to linear splines plus interactions with categorical features in a linear model. The knot placement is the hard part, but you can try greedy approaches or heuristics. I think this is closer to what you want. DTs can cope with mixed categorical/numerical features though, so that's a non-issue. $\endgroup$ – Firebug Aug 29 '18 at 11:46
1
$\begingroup$

Welcome to CV. As far as I know, every tree method can use both categorical and continuous variables. Certainly PROC HPSPLIT in SAS can do so, as can the party and rpart packages in R.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.