3
$\begingroup$

1.AdaBoost updates the weight of the sample By the current weak classifier in training each stage. Why doesn't it use the all of the previous weak classifiers to update the weight. (I had tested it that it converged slowly if I used the previous weak classifiers to update the weight )

2.It need to normalize the weight to 1 after updating(just need to multiply factor). I think this step can be omitted in implementing. Right?

$\endgroup$
2
$\begingroup$

2.It need to normalize the weight to 1 after updating(just need to multiply factor). I think this step can be omitted in implementing. Right?

Normalizing the sample weight is useful to prevent numerical instability, but is not always mentioned in sources.

  • The summary of the AdaBoost M1 algorithm in the well known book "Elements of Statistical Learning" on page 339 does not include normalization of weights.

In practice it is important however, and the implementation in Sklearn does normalize the weights between each boost iteration.

See line 164 - 166 in the source code

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

Answer to #1: In iteration $t$ the weight already carries in it contributions from the weak hypothesis generated in iterations $1,\ldots,t-1$: they already made their contribution, so there's no need to look at them again in iteration $t$.

in other words: the weight of sample $i$ counts, in a certain sense, how many of the previous weak hypothesis failed to correctly classify sample $i$. When we get to iteration $t$ the contribution of the previous weak hypothesis was already counted.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Yes, And I found that If I count the Previous again. It cann't be converged $\endgroup$ – Samuel May 27 '13 at 1:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.