2
$\begingroup$

I am currently searching for the advantages of random forest over decision trees, but unfortunately I didn't find a research paper that does such a conclusion that summarize all the advantages of RF over DT. Does anyone know any trusted resource that I can use to find a complete answer?

Thanks in advance

Improve this question
$\endgroup$

3 Answers 3

Reset to default
1
$\begingroup$

Briefly, although decision trees have a low bias / are non-parametric, they suffer from a high variance which makes them less useful for most practical applications.

By aggregating multiple decision trees, one can reduce the variance of the model output significantly, thus improving performance. While this could be archived by simple tree bagging, the fact that each tree is build on a bootstrap sample of the same data gives a lower bound on the variance reduction, due to correlation between the individual trees. Random Forest addresses this problem by sub-sampling features, thus de-correlating the trees to a certain extend and therefore allowing for a greater variance reduction / increase in performance.

Details on this can be found in chapter 15 of The Elements of Statistical Learning as well as chapter 8 of An Introduction to Statistical Learning by Hastie and Tibshirani.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you for this extremely informative and summarizing of the key advantages. I believe it will help me while I am searching. I have one question about "non-parametric" point, how this is a disadvantage in DT? $\endgroup$
    – CS Student
    Commented Apr 3, 2019 at 7:27
  • $\begingroup$ Please if you have a resource support your answer provide it at the end of your answer so I can choose it as the best answer for the question. Thank you very much. $\endgroup$
    – CS Student
    Commented Apr 3, 2019 at 10:11
  • 1
    $\begingroup$ @CSStudent Having a low bias / being non-parametric is not a disadvantage at all, however non-parametric models tend to have a high variance and often over-fit easily. Bagging (of which Random Forests are a special case in context of decision trees) trys to reduce the variance, thus making models more robust. In theory, every model can be bagged, it just happens to work particularly well for trees because they have an exceptionally high variance. $\endgroup$
    – Scholar
    Commented Apr 3, 2019 at 11:22
  • $\begingroup$ I couldn't describe how much you made this post a good source of information. Thank you very much for your efforts, I wish that I can evaluate this answer. Thank you once again. $\endgroup$
    – CS Student
    Commented Apr 3, 2019 at 14:33
  • 1
    $\begingroup$ @CSStudent no problem, thanks for your kind words. $\endgroup$
    – Scholar
    Commented Apr 3, 2019 at 15:05
3
$\begingroup$

Another excellent resource is the book by Max Kuhn and Kjell Johnson, Applied Predictive Modeling. The book focuses more on the practice of modeling and less on theory, so it might be a good complement to the two citations from above. A nice feature of the book is that it provides a summary of various models (including tree-based approaches) and some of their characteristics with respect pre-processing, interpretability, tuning parameters, computation time etc.

Improve this answer
$\endgroup$
0
3
$\begingroup$

Detailed explanations of the random forest procedure and its statistical properties can be found in Leo Breiman, "Random Forests," Machine Learning volume 45 issue 1 (2001) as well as the relevant chapter of Hastie et al., Elements of Statistical Learning.

OP's username is "CS Student," which suggests that OP has access to a university library. Librarians are extremely helpful people! Their job is to organize and retrieve knowledge! A librarian probably won't do your searching for you, but they might be able to help you learn how the library's collection is organized and how to use different research tools.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.