0
$\begingroup$

I am following this video tutorial to understand Adaboost

I am confused about the sample weights updating. It first calculates the amount of say of each stump by this formula,

enter image description here

where total error is sum of the weights associated with incorrectly classified samples.

Based on this formula, amount of say ranges from negative infinity to positive infinity. Then it updates the sample weight according to this rule.

  1. Incorrectly classified samples

enter image description here

  1. Correctly classified samples

enter image description here

I am confused about why this updating works. I understand when amount of say is greater than 0, the formula makes sense to me. But when amount of say is below 0, then we would actually decrease the weights for incorrectly classified samples, and increase the weights for correctly classified samples. Anything I miss here? How to udpate sample weights each round?

$\endgroup$

1 Answer 1

0
$\begingroup$

In Adaboost you always decrease the importance of correctly classified samples, and increase the importance of the incorrectly classified ones. The exponential function has values >1 for positive exponent and between 0 and 1 for a negative one.

enter image description here

So assuming "amount of say" is a positive factor, incorrectly classified samples will have their weights multiplied with factor >1, while correctly classified ones will be reduced, with a factor from 0 to 1.

$\endgroup$
2
  • $\begingroup$ But the formula does not work when amount of say is a negative factor. When amount of say is negative, based on the formula here, sample weights for incorrectly classified samples actually decrease, but for correctly classified samples increase. $\endgroup$
    – ycenycute
    Nov 5, 2021 at 0:08
  • $\begingroup$ Or shall we use another set of formula when amount of say is negative? $\endgroup$
    – ycenycute
    Nov 5, 2021 at 0:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.