Benchmark data for Random Forest evaluation I would like to make/explore several alternative formulations of random forest (link) and am looking for the current state of the science for evaluating performance.
In the reference file Leo provides "microarray", "DNA", "glass", and "spectral" data to exhibit the performance of his method.  Are these available?  How do I get them in order to first reproduce the results, and explore performance of changed parameter settings?  How general and authoritative are they? 
Other references that I have thus far found include (this), (this), and (this).  Can you suggest rigorous, modern, and relatively complete list of quality classification benchmarks to be used to compare the performance of very similar random forests?
 A: I think random forests are still mostly used in the form they were introduced by Breiman in his 2001 paper. There have been some attempts to improve them by e.g. moving beyond majority voting (http://link.springer.com/chapter/10.1007/978-3-540-30115-8_34), but my impression that this stuff isn't main-stream practice. You can find a good recent review of random forests in Elements of Statistical Learning (http://www-stat.stanford.edu/~tibs/ElemStatLearn/).
The datasets used by Breiman can be found at http://archive.ics.uci.edu/ml/. These datasets are well known classics. The downside is that they are not very large compared to some other datasets out there. That being said, I think the UCI datasets are a great place to start your investigations.
Finally - I think there's still a lot of good work to be done on random forests; the field is far from complete. Good luck!
A: One very relevant paper is Fernández-Delgado, Cernadas, Barro & Amorim, "Do we Need Hundreds of Classifiers to Solve Real World Classification Problems?", JMLR, 2014. The authors evaluated many classifiers, among them multiple versions of Random Forests, on the entire UCI repository as of that time and find that Random Forest variants indeed perform best. It seems like specific variants of Random Forests may work better for specific classes of problems, but overall, plain vanilla Random Forests work very well indeed.
Of course, the UCI repository has grown from the 121 datasets the authors used to (currently) 394 datasets (although probably not all of these are classification), so it might make sense to update that study.
