Adaboost prediction is the sign of the strong classifier. How can we obtain the probability of the prediction $P(y = 1 | x)$?

Can we use the logistic function or some other function as follows:

$$P(y = 1 | x) = \frac1{1+\exp(-F(x))}$$

where $F(x)$ is the strong classifier.

  • 2
    $\begingroup$ Why not use gradient boosting? There is rarely a reason to use adaboost instead in 2018. $\endgroup$ – Matthew Drury Feb 17 '18 at 20:51
  • 1
    $\begingroup$ Do you mean gradient boosting with exponential loss function? In that case how to convert the score into probabilities? $\endgroup$ – gnikol Feb 27 '18 at 17:02

You can do something similar, mathematically, but with a slightly different sigmoidal function to what you specified. To convert the output of AdaBoost

$$F(x) = \sum_{t=1}^T \alpha_t h_t (x)$$

to a conditional probability, you can pass it through the following sigmoidal:

$$\pi(x) = \frac{1}{1+e^{-2F(x)}}$$

(Source, including reasoning: Schapire-Freund Section 7.5.3.)

But beware: the probabilities will be inaccurate on small data sets because of the inherent assumptions of the AdaBoost model.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.