# Regression trees with multiple input and output levels

I am looking for different modelling approaches which are able to build regression trees (i.e. with continuous input and output variables) with multiple input and output levels.

The most common approach (e.g. with CART) is to recursively do binary splits. I am looking for methods that could split the target variable into multiple branches (according to some sensible metric) within every step.

Any hints, i.e. papers, links etc. are very welcome.

(The reason for me asking this question is that I want to find a good method to extend my OneR package for solving regression problems.)

Edit
To clarify: I am looking for a method which could split the respective input variable into $n$ intervals where each interval leads to an interval in the target variable. I guess you would need some constraints (e.g. the max number of intervals) to get useful results.

Edit 2
To clarify further just an illustrative example for one input and the target variable:

If x = (-5.01,-3.45] then y = (-1.06,1.06]
If x = (-3.45,-1.85] then y = (1.06,3.88]
If x = (-1.85,-0.25] then y = (-1.06,1.06]
If x = (-0.25,1.38]  then y = (-3.88,-1.06]
If x = (1.38,3.01]   then y = (-1.06,1.06]


An algorithm for just one input variable would be sufficient for my purposes (so I only need a one-step regression tree).

• Not sure I understand the question. It sounded like you were asking about $n$-ary splits rather than binary splits, but then you mentioned splitting the target variable. Is that right--do you mean somehow splitting the target (i.e. output) rather than (or jointly with) the input? Why do this rather than the usual approach of taking the mean target vector in each leaf node (which handles multiple outputs)? Apr 20, 2018 at 11:47
• @user20160: I edited the question - does this answer your question? Apr 20, 2018 at 12:45
• An example could help to properly understand (and answer) the question. At the moment, the comments by @vonjd still make sense. Apr 20, 2018 at 13:51
• @MichaelM: I gave an illustrative example - is this clearer now? Apr 20, 2018 at 17:11

I had a look at your work on the OneR package, witch is neat.

I understand the optbin function (OneR package) discretizes the explanatory variables by optimizing the predictive power of the resulting qualitative variable (eventually adding some penalization). Is that right ?

Then the optimization in OneR is made one explanatory variable at a time.

For the regression problem, I understand one issue is to jointly estimate the splits for the target, and the splits for the explanatory variable. It may be a good idea to look at methods that first, turn a regression problem into a classification (ordinal classification) problem.

• Thank you. Which methods are you referring to? Apr 26, 2018 at 15:05
• I was thinking about ordinal logistic regression (see for instance the polr from package mass). May 7, 2018 at 18:10