On dumping the xgboost (with objective of multi:softprob) for binary classification, I see booster dumps as: class0_booster0, class1_booster0, class0_booster1, class1_booster1, etc. I have noted down all the leaf values for class 0. How do I, then, calculate the probability for class 0? (I've tried 1/1+exp(-S), where S is the sum of all leaves, but had no luck.)

up vote 0 down vote accepted

Finally, could figure out! So, needed to do the following steps (per feature vector):

  • Get all the leaf values for all the boosters (both classes). Note that if the feature-attribute value is a zero, you'd have to take the child under missing=<> (by default xgb considers zeros as 'missing')
  • Sum up all the leaf values for each class separately: call them sum1, sum2 resp.
  • Add the bias_score to each sum (by default, it is 0.5): sum1 += 0.5, sum2 += 0.5.
  • Then apply the multi-class Softmax function to get the final probabilities for the two classes! i.e. exp(sum1)/(exp(sum1)+exp(sum2)), likewise, exp(sum2)/(exp(sum1)+exp(sum2)) respectively.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.