I am quite a newbie in this area:

  • What are the boosting methods for regression systems? I know about Gradient boosting; are there any other approaches?
  • Are there textbooks or tutorials devoted to this area?

While I haven't seen anything specifically, I doubt that it would achieve much, at least for linear regression. Each regression equation is just a linear combination of the predictors: $$\hat{y}^{(j)} = \sum_i \hat{\beta}_i^{(j)} x_i$$ Most boosting algorithms in turn combine multiple predictors by taking a weighted average, which is another linear combination: $$\hat{y} = \sum_j \hat{w}^{(j)} \hat{y}^{(j)}$$ So the final output is still a linear combination of the predictors, only with the coefficients obtained by a complicated process.

| cite | improve this answer | |
  • $\begingroup$ yeah but it might do well for logistic regression. $\endgroup$ – highBandWidth Sep 8 '11 at 3:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.