I'm trying to calibrate some probabilities returned by different classifiers. I have plotted the auc and logloss of each one and I remark that the logloss is always improved (auc does not chance because I use monotonic function for the calibration).

Is there a result saying "that calibration always improve logloss score" or did I have those score enhancement by chance ?

  • 2
    $\begingroup$ Well, if your model optimizes logloss (e.g. logistic regression), then changing the solution makes logloss worse. $\endgroup$
    – Michael M
    Commented Sep 17, 2019 at 16:31
  • $\begingroup$ Seems legit indeed. Thank you :) $\endgroup$
    – Samos
    Commented Sep 18, 2019 at 10:56
  • $\begingroup$ Do you know if all the models that optimizes logloss return well calibrated probabilities ? $\endgroup$
    – Samos
    Commented Sep 18, 2019 at 11:05


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.