I know this might be a newbie question, but I'm trying to make sense of this sentence. I understand that these are several algorithms that together give a good result. But I don't understand how this can be implemented and why it is called an ensemble method, perhaps because of consistency?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
One of the best ways to understand ensemble method (in my humble opinion) is a Random Forest.
One of the first machine learning algorithms you learn is a "Decision Tree", which is just a flowchart i.e we answer a lot of yes/no questions and end with a conclusion based on the answers e.g:
is the person taller than 170? (Yes) Does the person weight more than 80 kg (yes) Does the person prefer wearing dresses? (No)
Based on that we might guess that the person is a man.
The problem with Decision Trees is that they (often) have a very high variance i.e if we vary some of the inputs a tiny bit, we might end up with a completely different answer - you can see this as a single person's personal opinion/guess.
We can reduce this variance by creating a lot of Decision Trees (many trees = forest) in a special "random" way (Random Forest), say 100 trees. You can view this as instead of asking only one person the questions we stated above, we ask 100 person's and take the majority vote e.g say 70 predict "male" and the remaining 30 predict "female" we then guess that the input is a "male" (in a regression setting we would take the average of the guesses). In that way we reduce the variance and get a more robust classifier.
The word "ensemble" is french and just mean "together" or "collection" - which is what we have here; a collection of decision trees.
Note here that the ensemble (Random Forest) consist of the same class of classifier i.e decision-trees but that does not have to be the case. You could create an ensemble classifier which consists of a decision tree, a neutral network and support vector machine (SVM), and then take the majority vote of those three.
