6
$\begingroup$

As I am only familiar with the basics regarding decision trees I would like to ask, with the risk of stating silly question: Is it possible to perform recursive partitioning with the group median as the response/objective?

For example, in stead of R's rpart() using means, could a similar tree be created with medians?

I want to do this because the continuous dependent variable I want to examine has a series of outliers that clearly affect the mean values (especially when the number of observations in each node gets smaller). Am I on the right track, or should I be using other kind of methods? Would preprocessing the data be another alternative (perhaps "capping" the values at a upper limit)?

$\endgroup$
2
  • 1
    $\begingroup$ This question is probably relevant, as is the fact that you can specify your own splitting criteria in rpart. See the discussion of the method argument in ?rpart. $\endgroup$
    – joran
    Aug 25, 2011 at 20:23
  • $\begingroup$ @joran: interesting, does anybody have experience with editing this method? $\endgroup$
    – Figaro
    Aug 26, 2011 at 6:46

3 Answers 3

4
$\begingroup$

You could also pre-process your data, using a transformation like the spatial sign transformation, or the rank-order transformation to minimize the impact of outliers.

$\endgroup$
2
  • 1
    $\begingroup$ Does a spatial sign transformation really remove outliers? $\endgroup$
    – Figaro
    Aug 26, 2011 at 21:10
  • 2
    $\begingroup$ @Figaro I believe it greatly reduces their affect. ncbi.nlm.nih.gov/pubmed/16711760 $\endgroup$
    – Zach
    Aug 26, 2011 at 21:13
3
$\begingroup$

Although I have never used it, the quantregForest package seems to do what you want.

Here is the description:

Quantile Regression Forests is a tree-based ensemble method for estimation of conditional quantiles. It is particularly well suited for high-dimensional data. Predictor variables of mixed classes can be handled. The package is dependent on the package randomForests, written by Andy Liaw.

There is also an article accompanying the quantregForest package:

Meinshausen N (2006). “Quantile Regression Forests.” Journal of Machine Learning Research, 7, 983–999.

$\endgroup$
2
  • $\begingroup$ What in the description makes you say that? Is it that random forest methods are generally more stable or is there something specific about this approach that makes it more suitable? $\endgroup$
    – Figaro
    Aug 26, 2011 at 21:17
  • $\begingroup$ @Figaro Typically, we model the mean of the response; quantile regression methods allow you to model, well, quantiles (e.g. the median), which generally entails a somewhat more robust loss function than squared error. $\endgroup$
    – joran
    Aug 27, 2011 at 20:06
1
$\begingroup$

In addition to Johannes's suggestion about quantregForest, there is also an R package called gbm (generalized boosted machine) which uses trees to calculate conditional quantiles.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.