How does Adaboost select best features from the sample data (or a unit feature vector)?

It would be nice if someone can explain if the above statement is true or not.

I've seen the term features and classifiers being used in place of each other in documentation that's available of Adaboost (that's what I thought).

For example: www.cs.nyu.edu/~eugenew/publications/viola-facedet04-talk.pdf

Please explain!


Well, first of all in the presentation you mentioned they just used a value of one feature is larger/smaller than some threshold (i.e. decision tree of a depth 1) as a partial classifier, thus this feature-classifier ambiguity.

Going back to the question, there are numerous way to get feature ranking from a boosting -- starting from counting how deep in boost structure the feature lies up to some permutation tests. In the work you quoted there is in fact no feature selection -- in training they just select best feature/threshold pair, in prediction they compute features "on demand" while the prediction goes through a boost to save computational time.

  • 1
    $\begingroup$ Can you give me some pointers to any existing implementation of adaboost or a paper that does "feature ranking" and training in the manner that you've mentioned? $\endgroup$ – garak Apr 9 '12 at 15:49
  • $\begingroup$ @raul_w Check out R package ada's varplot and gbm's summary method of gbm objects. They have nice citations in the manuals. $\endgroup$ – user88 Apr 10 '12 at 11:16
  • $\begingroup$ I guess the confusion comes from many people describing Adaboost along with Haar features as classifiers. I found that there are very few intro slides on adaboost as an individual concept (or with some other classifier other than haar like decision trees). $\endgroup$ – garak Apr 10 '12 at 14:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.