I am having difficulty understanding the derivation of the AdaBoost.R2 algorithm (AdaBoost for regression problems), as presented in this paper by Drucker (page 2), which seems to be the source that people cite when referring to it. Notably, sklearn's (the python library) implentation is based on this paper. He says it is a modification of the AdaBoost.R algorithm proposed by Freund and Schapire here (page 136).
There are 2 questions I have:
- Why is the final prediction of AdaBoost.R2 the weighted median of all of weak learners? This is in contrast to AdaBoost for classification problems, which takes a weighted average.
- It is my understanding (from section 10.4 in Hastie's Elements of Statistical Learning) that AdaBoost for classification can be understood as gradient boosting (Hastie, algorithm 10.2) with the exponential loss function $L=\exp(-y \,f(x))$, where $y$ is the target class and $f(x)$ is our prediction. It is a straightforward exercise to show that the AdaBoost.M1 algorithm (algorithm 10.1 in Hastie) follows from this approach. AdaBoost for regression does not use the exponential loss function, but can it, too, be derived in a similar manner?
Finally, in general, if there are any other sources that go over AdaBoost's implementations for regression problems (not necessarily just for regression trees), I would appreciate any links.