18
$\begingroup$

Is ER more efficient implementation (somelike Extreme Gradient Boosting is to gradient boosting)-- is the difference important from practical point of view ? There is R package which implements them. Is it new algorithm which overcomes "generic" implementation (RandomForest package from R) not only in terms of efficiency or also in some other areas?

Extreme Random Forest http://link.springer.com/article/10.1007%2Fs10994-006-6226-1

$\endgroup$
20
$\begingroup$

This is pretty simple -- RF optimizes splits on trees (i.e. select those which give best information gain with respect to decision) and ERF makes them at random. Now,

  • optimisation costs (not much, but still), so ERF is usually faster.
  • optimisation may contribute to correlation of trees in ensemble or overall overfitting, so ERFs are probably more robust, especially if the signal is weak.

Going even further in this direction, you can gain extra speed by equalising splits on each tree level, this way converting trees into ferns, which are also pretty interesting; there is my R implementation of such an individuum.

$\endgroup$
  • $\begingroup$ That link is broken, use CVLAB: Ferns $\endgroup$ – smci Jun 9 '15 at 21:29
  • $\begingroup$ I suppose that trees created by ERF are much larger then those of RF, because RF uses optimization which compress knowlege from dataset onto smaller trees $\endgroup$ – Qbik Jun 3 '16 at 7:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.