3
$\begingroup$

If I have

a set RF of Decision Trees trained using a Random Forest algorithm and

a set AB of Decision Trees (Stumps), trained using Adaboost

do I see it correctly that I can uniformly implement both Ensemble methods, by something along the line of

  double ensemble(Trees, weights) {
      dec = 0
      for T in Trees:
        dec += weights[T] * T.getDecision() //-1 for False, +1 for True
      return dec
  }

and call it for the Random Forest using

ensemble(RF, [1,1,...])

and for Adaboost using

ensemble(RF, w)

where w is the weight vector obtained form the Adaboost training.

I.e., after training the only difference between Adaboost and RandomForest is that Adaboost uses a weighted sum of the decision but RandomForest doesn't?

$\endgroup$
3
$\begingroup$

I believe that should be possibly. Basicly you need both adaboost and random forest to not only classify, but also quantify some kind of certainty. For RF, probabilistic voting is the average vote over trees. I see adaBag package for R outputs a similar probabilistic measure.

citing from package manual, "predict.boosting: Predict from a fitted boosting object."

prob a matrix describing, for each observation, the posterior probability or degree of support of each class. These probabilities are calculated using the proportion of votes in the final ensemble.

Use ROC plots to check if combined ensembles actually are better.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.