Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
1.AdaBoost updates the weight of the sample By the current weak classifier in training each stage. Why doesn't it use the all of the previous weak classifiers to update the weight. (I had tested it that it converged slowly if I used the previous weak classifiers to update the weight )
2.It need to normalize the weight to 1 after updating(just need to multiply factor). I think this step can be omitted in implementing. Right?
2.It need to normalize the weight to 1 after updating(just need to multiply factor). I think this step can be omitted in implementing. Right?
Normalizing the sample weight is useful to prevent numerical instability, but is not always mentioned in sources.
In practice it is important however, and the implementation in Sklearn does normalize the weights between each boost iteration.
See line 164 - 166 in the source code
Answer to #1: In iteration $t$ the weight already carries in it contributions from the weak hypothesis generated in iterations $1,\ldots,t-1$: they already made their contribution, so there's no need to look at them again in iteration $t$.
in other words: the weight of sample $i$ counts, in a certain sense, how many of the previous weak hypothesis failed to correctly classify sample $i$. When we get to iteration $t$ the contribution of the previous weak hypothesis was already counted.
