I have read Alexandru Niculescu-Mizil and Rich Caruana's paper "Obtaining Calibrated Probabilities from Boosting" and the discussion in this thread. However, I am still having trouble understanding and implementing logistic or Platt's scaling to calibrate the output of my multi-class boosting classifier (Gentle-boost with decision stumps).
I am somewhat familiar with generalized linear models, and I think I understand how the logistic and Platt's calibration methods work in the binary case, but am not sure I know how to extend the method described in the paper to the multi-class case.
To make things maybe even worse, the classifier I am using outputs only the following:
- $f_i$ = Number of votes that the classifier casts for the winning class for the sample $i$ that is being classified
- $y_i$ = Estimated class
At this point I have the following questions:
Q1: Do I need to use multinomial logit's to estimate probabilities? or can I still do this with logistic regression (e.g. in a 1-vs-all fashion)?
Q2: How could I define the intermediate target variables in Platt's scaling for the multi-class case?
Q3: I understand this might be a lot to ask, but would anybody be willing to sketch out a solution to this problem in Matlab?