Why is it that AdaBoost uses decision stumps for the weak learners? It seems simpler to me to just use the weighted majority of the data points for the classification. Why shouldn't we do this?

  • $\begingroup$ How does predicting the majority class of the data use the features at all? $\endgroup$ – Sycorax Apr 20 '20 at 3:15
  • $\begingroup$ The weights will be updated each iteration. So if a point is wrongly classified by the majority class it will be more likely to end up the majority class in the next iteration. After enough iterations wouldn't the weights cause most points to be correctly classified? $\endgroup$ – MLNewbie Apr 20 '20 at 3:56
  • $\begingroup$ That's a more detailed explanation of what you proposed in your question, but it doesn't appear to use the features in any way. To put it another way, if you think the overall class label is more informative than any of your features, why did you collect features at all? Intuitively, I think this is just a more ornate way to create a random-walk behavior for the weights because there's no signal that's attached to the observations themselves because the features are always ignored. $\endgroup$ – Sycorax Apr 20 '20 at 15:51
  • $\begingroup$ But you can easily test this: code this up and try it out. I think you'll find that it quickly just oscillates around random guessing, $\endgroup$ – Sycorax Apr 20 '20 at 15:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.