On dumping the xgboost (with objective of multi:softprob) for binary classification, I see booster dumps as: class0_booster0, class1_booster0, class0_booster1, class1_booster1, etc. I have noted down all the leaf values for class 0. How do I, then, calculate the probability for class 0? (I've tried 1/1+exp(-S), where S is the sum of all leaves, but had no luck.)
Finally, could figure out! So, needed to do the following steps (per feature vector):
- Get all the leaf values for all the boosters (both classes). Note that if the feature-attribute value is a zero, you'd have to take the child under missing=<> (by default xgb considers zeros as 'missing')
- Sum up all the leaf values for each class separately: call them sum1, sum2 resp.
- Add the bias_score to each sum (by default, it is 0.5): sum1 += 0.5, sum2 += 0.5.
- Then apply the multi-class Softmax function to get the final probabilities for the two classes! i.e. exp(sum1)/(exp(sum1)+exp(sum2)), likewise, exp(sum2)/(exp(sum1)+exp(sum2)) respectively.
