Difference between Random Forests and Decision tree

Question

I was led to use some techniques of statistics and machine learning, especially random forest method.

I need to understand the difference between random forests and decision trees and what are the advantages of random forests compared to decision trees.

Answer 1

You are right that the two concepts are similar. As is implied by the names "Tree" and "Forest," a Random Forest is essentially a collection of Decision Trees. A decision tree is built on an entire dataset, using all the features/variables of interest, whereas a random forest randomly selects observations/rows and specific features/variables to build multiple decision trees from and then averages the results. After a large number of trees are built using this method, each tree "votes" or chooses the class, and the class receiving the most votes by a simple majority is the "winner" or predicted class. There are of course some more detailed differences, but this is the main conceptual difference.

Answer 2

When using a decision tree model on a given training dataset the accuracy keeps improving with more and more splits. You can easily overfit the data and doesn't know when you have crossed the line unless you are using cross validation (on training data set). The advantage of a simple decision tree model is easy to interpret, you know what variable and what value of that variable is used to split the data and predict outcome.

A random forest is like a black box and works as mentioned in above answer. It's a forest you can build and control. You can specify the number of trees you want in your forest(n_estimators) and also you can specify max num of features to be used in each tree. But you cannot control the randomness, you cannot control which feature is part of which tree in the forest, you cannot control which data point is part of which tree. Accuracy keeps increasing as you increase the number of trees, but becomes constant at certain point. Unlike decision tree, it won't create highly biased model and reduces the variance.

When to use to decision tree:

1. When you want your model to be simple and explainable
2. When you want non parametric model
3. When you don't want to worry about feature selection or regularization or worry about multi-collinearity.
4. You can overfit the tree and build a model if you are sure of validation or test data set is going to be subset of training data set or almost overlapping instead of unexpected.

When to use random forest :

1. When you don't bother much about interpreting the model but want better accuracy.
2. Random forest will reduce variance part of error rather than bias part, so on a given training data set decision tree may be more accurate than a random forest. But on an unexpected inference data set, Random forest always wins in terms of accuracy.

Answer 3

The random forest algorithm is a type of ensemble learning algorithm. This means that it uses multiple decision trees to make predictions. The advantage of using an ensemble algorithm is that it can reduce the variance in the predictions, making them more accurate. The random forest algorithm achieves this by averaging the predictions of the individual decision trees.

The decision tree algorithm is a type of supervised learning algorithm. This means that it requires a training dataset in order to learn how to make predictions. The advantage of using a supervised learning algorithm is that it can learn complex patterns in the data. The disadvantage of using a supervised learning algorithm is that it takes longer to train than an unsupervised learning algorithm.