Got a good Random Forest predictor. What's next? Okay, I have created a Random Forest, I get a very good prediction of response. What's next? How can I improve it further?
I ask this because I often see competitions in Kaggle, where RF is used as a benchmark, and many people manage to obtain scores much higher than that of the RF based solution. In some cases, the data is also unlabelled, preventing you to apply manual judgment.
How do they beat RF by such decent margins?
 A: The key is to build more models.  If read solutions papers for competitions like the Netflix prize, the winning team had something like 100+ models doing the prediction.  They then combined the models linearly (i.e. regress the actual values against the 100 or so predicted values) to get weights for individual models.  There were maybe four or five different general types of models, and then the rest were variations on those models with different features or with different tuning parameters.
Though it's important to note that each additional model gives you significantly decreasing returns.  I think the winners of the Netflix prize found that 16 models got them to within a few percent of their final RMSE, meaning that the other ~80 some models contributed tinier and tinier amounts of improvement to the RMSE.
A: RF can be tune by different parameters.
The most important parameter is the number of variables randomly sampled. The author recommend different values for regression and classification. But it is worth to tune by your own problem and data set.
The size of terminal nodes of trees may also affect the performance. Similar to above, there are default values recommended by the author, but we can tune it.
The number of trees may be tuned if needed, but RF is seldom overfitted. So this may not affect too much if the number of trees is large enough.
Well, different model has different advantages and different strong application which they are suitable. This depends on the characteristics of your data set, some other special models are likely to better than RF in some specific topics. But RF is easy to tune, and it is usually reasonably good model, generally speaking, even by default setting.
