Meaning of Bagged Random Forests? I'm reading a paper that says that the authors used "bagged random forests". I couldn't understand this because as far as I know a random forest is a kind of bagging on its own. So a random forest is a bag of trees. But a bagged random forest?! Would that mean a bag of random forests with each random forest having 10 or 100 trees?!
This is the paper: http://users.cs.fiu.edu/~lzhen001/activities/KDD_USB_key_2010/docs/p243.pdf

Also

 A: As a colleague of the authors, I can address this question.


*

*To directly answer the OP, @rapaio is correct: the top quotation means that the authors created 10 separate bags each with a random forest of 10 trees -- there will be 100 total trees.

*As @rapaio mentioned, there's no clear cut reason why this performed better than 100 bags or 100 random forest. The Weka implementation made it easy to perform all five experiments experiments (single tree, 10 bags, 100 bags, 100 random forest, 10 bags of 10 random forests) and the authors felt the result was interesting enough to mention. Whether this is something that shows up as a general trend against other datasets (such as Kaggle or UCI) could be the basis for a good research paper.

*@rapaio did not mention it, but there may also be some interplay between the various oob estimates and the final performance metric -- AUC (not accuracy). Perhaps this is a trend only observed when optimizing for rank-order and not necessarily discrete predictions.

A: I would venture that it refers to regular random forests, but the author wants to bring out the distinction between a) the bagging / bootstrapping of the observations used for each tree and b) the random selection of a subset of the input parameters. not sure though.
A: I tried to understand why bagging 10 random forests would work better than a random forest with 100 tress and I see no rational reason. I do not exclude that there could be some Weka implementation details.
However to answer your question I believe it is talking about a bagging ensemble with 10 bags and in each bag a random forest with 10 trees.
I do not have a proof, but there are some elements which I consider it provides enough evidence for this:


*

*In the first paragraph, the last sentence which starts with "we found that neither 100.." it talks about building either a bagging with 100 trees or a random forest with 100 trees. Now 100 is 10 x 10, which are 10 bags x a random forest with 10 trees. 

*10 bagged random forests means the same thing from the previous point (even if it is not usual to bag random forests)

*in Weka is very easy to combine in this way some classifiers, since they can be chained together due to the fact that they implement the same interface (this is a programming stuff, I am a programmer, and I confirm that it is obvious that Weka intended to make such kind of experimenting as easy as possible)

*there is a way to decrease the total compute out-of-the-bag error if you group 10 random forests with 10 trees into bagging, instead of 100 random forest. If you have 10 bags and in each bag one rf with 10 trees, than the oob error is computed for the other instance of the 10 bootstrap samples from the bags. These 10 oob samples sets are classified with a random forest instead of a normal random tree. Of course, the random forest has in general less variability, so the oob errors from the 10 bags could be lower than if you have the average of oob errors for 100 oobs predicted with trees. I do not know if this is a real improvement, it might be. From my point of view it is not, it looks like an introduction of a optimistic bias in oob estimation, but this is only my intuition and since I don't have an extensive experience, I might be wrong. Even so, it looks like a plausible track

*last argument is a personal intuition: I asked myself why one would over-complicate these kind of formulations for nothing? I found reasonable to believe they did that with a purpose, and a "reasonable reason" is that they simply used a bagging of random forest.   

