n/m asymptotics and logistic regression goodness of fit tests

Question

in the case of logistic regression, say our model has $p$ covariates and we have $J$ distinct covariate patterns (distinct combinations of the values of the p covariates) and we have a total of $n$ observations. The number of covariate patterns with pattern $i$ is denoted by $m_i$. Then, let $y_j$ denote the number of success responses out of the $m_j$.

We have the statement:

The distribution of goodness of fit tests are obtained by letting $n \to \infty$. They are said to be $n$ asymptotic. If we fix $J<n$ and let $n \to \infty$ then each $m_j \to \infty$, distribution assumptions based on these are known as m-asymptotics.

I am trying to understand why the deviance and pearson chi squared statistics produce incorrect $p$-values in the case that $J \approx n$.

The text by Hosmer, Lemeshow states that when $J \approx n$, the distribution of both the pearson residual and the deviance residual is achieved under n asymptotics, and hence, the number of 'parameters' is increasing at the same rate as the sample size.

I don't quite understand why this is problematic, I'm looking for an intuitive explanation of this

Answer 1

To get the 'intuition' you might reason as follows:

I am taking $\chi^2$ as an example, and referring to the formulas (5.1) and (5.2) in this link, which is the book by Hosmer-Lemeshow.

The $\chi^2$ variable is defined as a sum of squared standard normal variables, so if you look at the formulas (5.1) and (5.2) it is assumed that $r(m_j, \hat{\pi}_j)=\frac{y_j - m_j \hat{\pi}_j }{\sqrt{m_j \hat{\pi}_j(1-\hat{\pi}_j)}}$ are standard normal variables.

Note that $y_j$ is the number of successes among the cases with the $j$-th covariate pattern, and $m_j$ is the number of cases with the $j$-th covariate pattern.

Now if you look at that $j$-th covariate pattern as a Binomial random variable with success probability $\pi_j$ and size $m_j$, then the mean of this random variable is (see properties of the Binomial variable) $\mu_j=m_j\pi_j$ and its standard deviation is $\sigma_j=\sqrt{m_j \pi_j (1-\pi_j)}$.

If the size ($m_j$) of this Binomial random variable is ''large'' then the Binomial variable is approximately normal with the same mean and variance.

So if all the $m_j$ are ''large'' then $\frac{y_j - m_j \pi_j}{\sqrt{m_j \pi_j (1-\pi_j)}}$ is approximately standard normal, and then, by definition of $\chi^2$, the sum $X^2=\sum_j \left( \frac{y_j - m_j \pi_j}{\sqrt{m_j \pi_j (1-\pi_j)}} \right) ^2$ has a $\chi^2$ distribution.

If you compare this to the formulas (5.1) and (5.2) in the link supra, you will see that the $\chi^2$ distribution in Hosmer-Lemeshow's book uses the above property, which only holds if $m_j$ are all large !

If $J \approx n$ then all $m_j$ are more or less equal to one, which is not ''large'', so if $J \approx n$ the variable $X^2$ defined in (5.1) and (5.2) can not be shown to follow a $\chi^2$ distribution.

The p-values are derived under the property that $X^2$ is $\chi^2$, so if you can not show this $\chi^2$ property, then the p-values derived from it are also 'doubtfull'.