Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Is ER more efficient implementation (somelike Extreme Gradient Boosting is to gradient boosting)-- is the difference important from practical point of view ? There is R package which implements them. Is it new algorithm which overcomes "generic" implementation (RandomForest package from R) not only in terms of efficiency or also in some other areas?

Extreme Random Forest

share|improve this question
up vote 13 down vote accepted

This is pretty simple -- RF optimizes splits on trees (i.e. select those which give best information gain with respect to decision) and ERF makes them at random. Now,

  • optimisation costs (not much, but still), so ERF is usually faster.
  • optimisation may contribute to correlation of trees in ensemble or overall overfitting, so ERFs are probably more robust, especially if the signal is weak.

Going even further in this direction, you can gain extra speed by equalising splits on each tree level, this way converting trees into ferns, which are also pretty interesting; there is my R implementation of such an individuum.

share|improve this answer
That link is broken, use CVLAB: Ferns – smci Jun 9 '15 at 21:29
@smci Fixed, thanks. – mbq Jun 9 '15 at 22:18
I suppose that trees created by ERF are much larger then those of RF, because RF uses optimization which compress knowlege from dataset onto smaller trees – Qbik Jun 3 at 7:41

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.