# Applicability of the Hosmer-Lemeshow test

Description

To test the goodness of fit of modelled event probabilities $$\bar{P}=\{P_{1},...,P_{g}\}\in(0,1)^{G}$$ for binomially distributed $$G\in\mathbb{N}^{+}$$ grades, I’ve applied the HL (Hosmer-Lemeshow) test as follows:

Note that for $$G$$ grades, $$\bar{P}$$ is initially based on a multivariate logistic regression and is monotonically increasing.

1. Let $$P_{g}$$ be the modelled event probability, $$N_{g}\in\mathbb{N}^{+}$$ the number of observations and $$O_{g}\in\mathbb{N}^{+}$$ the number of outcomes (naturally $$O_{g}\leq N_{g}$$) for grade $$g\in\{1,…,G\}$$

2. For each grade $$g$$ sample $$M\in\mathbb{N}^{+}$$ times $$\hat{O}_{g}\sim B(N_{g},P_{g})$$ and check (either visually or based on e.g. total variation distance) whether the HL test statistic $$H = \sum^{G}_{g=1}\Big(\frac{(O_{g}-N_{g}P_{g})^{2}}{N_{g}P_{g}}+\frac{(N_{g}-O_{g}-(N_{g}-N_{g}P_{g}))^{2}}{N_{g}-N_{g}P_{g}}\Big)$$ follows a $$\chi^{2}$$ distribution with $$G$$, $$G-1$$ or $$G-2$$ degrees of freedom for large enough $$M$$

3. Compute the p-value of $$H$$ based on $$\bar{O}=\{O_{1},...,O_{g}\}$$, $$\bar{N}=\{N_{1},...,N_{g}\}$$ and $$\bar{P}$$ using a $$\chi^{2}$$ distribution with $$G$$, $$G-1$$ or $$G-2$$ degrees of freedom depending on step 2

Question

Sampling in step 2 demonstrates that $$H$$ follows a $$\chi^{2}$$ distribution with $$G$$ degrees of freedom when $$M\rightarrow\infty$$ – which I expect as $$H$$ follows the sum of $$G$$ squared standard normal random variables – however, literature states that the expected degrees of freedom is either $$G-2$$ or $$G-1$$ in some special cases (Wikipedia and Surjanovic & Loughin, Oct/23 – most recent article I found containing all necessary references). I believe this is because the above test is different from the HL test; however, after setting $$\bar{P}$$ such that it follows the curve of a logistic regression according to $$P_{g}=\frac{1}{1+e^{-(g-\mu)/s}}$$ with $$G=10$$ and, location- and scale parameters $$\mu=5.5$$ and $$s=1$$, $$G$$ degrees of freedom still applies. How does the test above deviate from the HL test (as described on the Wikipedia page)?

• Thanks! In hindsight, my question was unclear, but it drove me to rebuild a HL test and concluded that if every sampled data set is fitted by a logistic regression, the HL test statistic follows $G-2$ degrees of freedom, however, if these probabilities are fixed (even when based on a logistic regression-like curve ) and multiple data sets are sampled based on these probabilities, the HL test statistic follows $G$ degrees of freedom even when using the characteristic deciles and/or groups according to the ("fitted") - here I concluded my question was unclear based on your answer - probabilities Commented Feb 23 at 10:16