2
$\begingroup$

I've only seen the Hosmer-Lemeshow test used for Logistic Regression. Why can't it be used for algorithm that produces probabilities for binary classification?

All the Hosmer-Lemeshow test requires probabilities and binary labels. So, why hasn't it been used on models such as support vector machines, random forest, and any other binary classification algorithm?

$\endgroup$
2
$\begingroup$

The Hosmer–Lemeshow_test is actually applying the Pearson's chi-squared test on deciles of confidence for both binary outcomes.

As such, it is can indeed be used in any binary classifier that provides confidence.

As for why it is more common for logistic regression, I can speculate. It seems that the test was introduced at Applied logistic regression. D. W. Hosmer and S. Lemeshow, Wiley, New York, 1989. As the book name implies, the focus is logistic regression so that is why it is used in this classifier context.

As for why it is not common when other classifier are being used, I believe it is since other measures suites more the researchers need in unsupervised learning. The chi-square test estimates fit and it is symmetric with respect to the deviation. In supervised learning we are usually interested in many aspects of the performance. One might want to maximize precision while other would like to maximize recall.

$\endgroup$
  • 1
    $\begingroup$ One might also point out that is has been re-evaluated by a group including the original authors here $\endgroup$ – mdewey Dec 22 '16 at 13:36
  • $\begingroup$ But all of the other measures of performance, such as ROC curve, F measure etc., are used for any kind of binary classification. but Hosmer-Lemeshow seems to be ONLY used for logistic regression. $\endgroup$ – Sother Dec 23 '16 at 6:30
  • 1
    $\begingroup$ You are right. I guess that the roots are due to history and not due to statistics. The other measures are much older and therefore more common (for all classifiers). $\endgroup$ – DaL Dec 25 '16 at 6:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.