3
$\begingroup$

I am looking for different modelling approaches which are able to build regression trees (i.e. with continuous input and output variables) with multiple input and output levels.

The most common approach (e.g. with CART) is to recursively do binary splits. I am looking for methods that could split the target variable into multiple branches (according to some sensible metric) within every step.

Any hints, i.e. papers, links etc. are very welcome.

(The reason for me asking this question is that I want to find a good method to extend my OneR package for solving regression problems.)

Edit
To clarify: I am looking for a method which could split the respective input variable into $n$ intervals where each interval leads to an interval in the target variable. I guess you would need some constraints (e.g. the max number of intervals) to get useful results.

Edit 2
To clarify further just an illustrative example for one input and the target variable:

If x = (-5.01,-3.45] then y = (-1.06,1.06]
If x = (-3.45,-1.85] then y = (1.06,3.88]
If x = (-1.85,-0.25] then y = (-1.06,1.06]
If x = (-0.25,1.38]  then y = (-3.88,-1.06]
If x = (1.38,3.01]   then y = (-1.06,1.06]

An algorithm for just one input variable would be sufficient for my purposes (so I only need a one-step regression tree).

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ Not sure I understand the question. It sounded like you were asking about $n$-ary splits rather than binary splits, but then you mentioned splitting the target variable. Is that right--do you mean somehow splitting the target (i.e. output) rather than (or jointly with) the input? Why do this rather than the usual approach of taking the mean target vector in each leaf node (which handles multiple outputs)? $\endgroup$
    – user20160
    Commented Apr 20, 2018 at 11:47
  • $\begingroup$ @user20160: I edited the question - does this answer your question? $\endgroup$
    – vonjd
    Commented Apr 20, 2018 at 12:45
  • 1
    $\begingroup$ An example could help to properly understand (and answer) the question. At the moment, the comments by @vonjd still make sense. $\endgroup$
    – Michael M
    Commented Apr 20, 2018 at 13:51
  • $\begingroup$ @MichaelM: I gave an illustrative example - is this clearer now? $\endgroup$
    – vonjd
    Commented Apr 20, 2018 at 17:11

1 Answer 1

Reset to default
1
$\begingroup$

I had a look at your work on the OneR package, witch is neat.

I understand the optbin function (OneR package) discretizes the explanatory variables by optimizing the predictive power of the resulting qualitative variable (eventually adding some penalization). Is that right ?

Then the optimization in OneR is made one explanatory variable at a time.

For the regression problem, I understand one issue is to jointly estimate the splits for the target, and the splits for the explanatory variable. It may be a good idea to look at methods that first, turn a regression problem into a classification (ordinal classification) problem.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you. Which methods are you referring to? $\endgroup$
    – vonjd
    Commented Apr 26, 2018 at 15:05
  • $\begingroup$ I was thinking about ordinal logistic regression (see for instance the polr from package mass). $\endgroup$
    – Ooona
    Commented May 7, 2018 at 18:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.