3
$\begingroup$

I am looking for different modelling approaches which are able to build regression trees (i.e. with continuous input and output variables) with multiple input and output levels.

The most common approach (e.g. with CART) is to recursively do binary splits. I am looking for methods that could split the target variable into multiple branches (according to some sensible metric) within every step.

Any hints, i.e. papers, links etc. are very welcome.

(The reason for me asking this question is that I want to find a good method to extend my OneR package for solving regression problems.)

Edit
To clarify: I am looking for a method which could split the respective input variable into $n$ intervals where each interval leads to an interval in the target variable. I guess you would need some constraints (e.g. the max number of intervals) to get useful results.

Edit 2
To clarify further just an illustrative example for one input and the target variable:

If x = (-5.01,-3.45] then y = (-1.06,1.06]
If x = (-3.45,-1.85] then y = (1.06,3.88]
If x = (-1.85,-0.25] then y = (-1.06,1.06]
If x = (-0.25,1.38]  then y = (-3.88,-1.06]
If x = (1.38,3.01]   then y = (-1.06,1.06]

An algorithm for just one input variable would be sufficient for my purposes (so I only need a one-step regression tree).

$\endgroup$
  • 2
    $\begingroup$ Not sure I understand the question. It sounded like you were asking about $n$-ary splits rather than binary splits, but then you mentioned splitting the target variable. Is that right--do you mean somehow splitting the target (i.e. output) rather than (or jointly with) the input? Why do this rather than the usual approach of taking the mean target vector in each leaf node (which handles multiple outputs)? $\endgroup$ – user20160 Apr 20 '18 at 11:47
  • $\begingroup$ @user20160: I edited the question - does this answer your question? $\endgroup$ – vonjd Apr 20 '18 at 12:45
  • 1
    $\begingroup$ An example could help to properly understand (and answer) the question. At the moment, the comments by @vonjd still make sense. $\endgroup$ – Michael M Apr 20 '18 at 13:51
  • $\begingroup$ @MichaelM: I gave an illustrative example - is this clearer now? $\endgroup$ – vonjd Apr 20 '18 at 17:11
1
$\begingroup$

I had a look at your work on the OneR package, witch is neat.

I understand the optbin function (OneR package) discretizes the explanatory variables by optimizing the predictive power of the resulting qualitative variable (eventually adding some penalization). Is that right ?

Then the optimization in OneR is made one explanatory variable at a time.

For the regression problem, I understand one issue is to jointly estimate the splits for the target, and the splits for the explanatory variable. It may be a good idea to look at methods that first, turn a regression problem into a classification (ordinal classification) problem.

$\endgroup$
  • $\begingroup$ Thank you. Which methods are you referring to? $\endgroup$ – vonjd Apr 26 '18 at 15:05
  • $\begingroup$ I was thinking about ordinal logistic regression (see for instance the polr from package mass). $\endgroup$ – Ooona May 7 '18 at 18:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.