Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote -1 down vote favorite

I'm wondering if there exist methods similar to one used in random forest algorithm - I mean taking simultaneously bootstrap sample and random subset of features, then building statistisal model. Have anyone took this approach for building set of regression models ? Is this approach ( random subsample plus random subset of features ) somehow universal ?

Edition : the question is all about the possibility of putting some other model in the place of classification tree in random forest, what is left is some kind of meta-algorithm (as bagging can be view as meta-algorithm) = bagging + random subsets of variables

share|improve this question
You can use random forest for regression - if by regression you mean a continuous target. – B_Miner Jan 8 at 23:44
3  
What you are probably looking for is known as random subspaces (RS). This is the universal approach that has been published even before the random forest (RF). However, RF proposed using out of the bag samples for building the ensemble. – soufanom Jan 9 at 3:57
@B_Miner yes I know it, the question is all about the possibility of putting some other model in the place of classification tree in random forest – Qbik Jan 10 at 13:07

1 Answer

up vote 0 down vote

I believe what you are looking for is known as "bagging", a portmanteau of "boostrap aggregation". Indeed, the Random Forest algorithm is sometimes referred to as the bagged trees algorithm. There are various implementations of this, for instance the R library CARET provides a framework for creating bagging models from any base model of your choice (such as SVM, logistic regression or whatever else you like).

share|improve this answer
1  
but bagging is only about sampling from training set but not from the set of variables – Qbik Jan 10 at 13:05
That's not the use of the term that I've encountered, which includes R packages for bagging and the journal articles written by the bagging method pioneer Leo Brieman. I'm sure people have done bagging without taking sub-samples of the variables, however the term refers in general to methods that take random samples of the training set and predictors, at least in my experience. Again I'd point out that the RF algorithm is also known as 'Bagged Trees' and this certainly includes, in general, sampling the variables. – Bogdanovist Jan 10 at 19:52

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.