Skip to main content
2 of 11
added 47 characters in body
Amelio Vazquez-Reina
  • 19.7k
  • 27
  • 81
  • 120

Logistic and Platt calibrations for multi-class classifiers

I have read Alexandru Niculescu-Mizil and Rich Caruana's paper "Obtaining Calibrated Probabilities from Boosting" and the discussion in this thread. However, I am still having trouble understanding and implementing logistic or Platt's scaling to calibrate the output of my multi-class boosting classifier (Gentle-boost with decision stumps).

I am somewhat familiar with generalized linear models, and I think I understand how the logistic and Platt's calibration methods work in the binary case, but am not sure I know how to extend the method described in the paper to the multi-class case.

To make things maybe even worse, the classifier I am using outputs only the following:

  1. $f_i$ = Number of votes that the classifier casts for the winning class for the sample $i$ that is being classified
  2. $y_i$ = Estimated class

At this point I have the following questions:

Q1: Do I need to use multinomial logit's to estimate probabilities? or can I still do this with logistic regression (e.g. in a 1-vs-all fashion)?

Q2: How could I define the intermediate target variables in Platt's scaling for the multi-class case?

Q3: I understand this might be a lot to ask, but would anybody be willing to sketch out a solution to this problem in Matlab?

Amelio Vazquez-Reina
  • 19.7k
  • 27
  • 81
  • 120