I am currently searching for the advantages of random forest over decision trees, but unfortunately I didn't find a research paper that does such a conclusion that summarize all the advantages of RF over DT. Does anyone know any trusted resource that I can use to find a complete answer?
Thanks in advance
Briefly, although decision trees have a low bias / are non-parametric, they suffer from a high variance which makes them less useful for most practical applications.
By aggregating multiple decision trees, one can reduce the variance of the model output significantly, thus improving performance. While this could be archived by simple tree bagging, the fact that each tree is build on a bootstrap sample of the same data gives a lower bound on the variance reduction, due to correlation between the individual trees. Random Forest addresses this problem by sub-sampling features, thus de-correlating the trees to a certain extend and therefore allowing for a greater variance reduction / increase in performance.
Details on this can be found in chapter 15 of The Elements of Statistical Learning as well as chapter 8 of An Introduction to Statistical Learning by Hastie and Tibshirani.
Another excellent resource is the book by Max Kuhn and Kjell Johnson, Applied Predictive Modeling. The book focuses more on the practice of modeling and less on theory, so it might be a good complement to the two citations from above. A nice feature of the book is that it provides a summary of various models (including tree-based approaches) and some of their characteristics with respect pre-processing, interpretability, tuning parameters, computation time etc.
Detailed explanations of the random forest procedure and its statistical properties can be found in Leo Breiman, "Random Forests," Machine Learning volume 45 issue 1 (2001) as well as the relevant chapter of Hastie et al., Elements of Statistical Learning.
OP's username is "CS Student," which suggests that OP has access to a university library. Librarians are extremely helpful people! Their job is to organize and retrieve knowledge! A librarian probably won't do your searching for you, but they might be able to help you learn how the library's collection is organized and how to use different research tools.
