1
$\begingroup$

in the case of logistic regression, say our model has $p$ covariates and we have $J$ distinct covariate patterns (distinct combinations of the values of the p covariates) and we have a total of $n$ observations. The number of covariate patterns with pattern $i$ is denoted by $m_i$. Then, let $y_j$ denote the number of success responses out of the $m_j$.

We have the statement:

The distribution of goodness of fit tests are obtained by letting $n \to \infty$. They are said to be $n$ asymptotic. If we fix $J<n$ and let $n \to \infty$ then each $m_j \to \infty$, distribution assumptions based on these are known as m-asymptotics.

I am trying to understand why the deviance and pearson chi squared statistics produce incorrect $p$-values in the case that $J \approx n$.

The text by Hosmer, Lemeshow states that when $J \approx n$, the distribution of both the pearson residual and the deviance residual is achieved under n asymptotics, and hence, the number of 'parameters' is increasing at the same rate as the sample size.

I don't quite understand why this is problematic, I'm looking for an intuitive explanation of this

$\endgroup$
2
$\begingroup$

To get the 'intuition' you might reason as follows:

I am taking $\chi^2$ as an example, and referring to the formulas (5.1) and (5.2) in this link, which is the book by Hosmer-Lemeshow.

The $\chi^2$ variable is defined as a sum of squared standard normal variables, so if you look at the formulas (5.1) and (5.2) it is assumed that $r(m_j, \hat{\pi}_j)=\frac{y_j - m_j \hat{\pi}_j }{\sqrt{m_j \hat{\pi}_j(1-\hat{\pi}_j)}}$ are standard normal variables.

Note that $y_j$ is the number of successes among the cases with the $j$-th covariate pattern, and $m_j$ is the number of cases with the $j$-th covariate pattern.

Now if you look at that $j$-th covariate pattern as a Binomial random variable with success probability $\pi_j$ and size $m_j$, then the mean of this random variable is (see properties of the Binomial variable) $\mu_j=m_j\pi_j$ and its standard deviation is $\sigma_j=\sqrt{m_j \pi_j (1-\pi_j)}$.

If the size ($m_j$) of this Binomial random variable is ''large'' then the Binomial variable is approximately normal with the same mean and variance.

So if all the $m_j$ are ''large'' then $\frac{y_j - m_j \pi_j}{\sqrt{m_j \pi_j (1-\pi_j)}}$ is approximately standard normal, and then, by definition of $\chi^2$, the sum $X^2=\sum_j \left( \frac{y_j - m_j \pi_j}{\sqrt{m_j \pi_j (1-\pi_j)}} \right) ^2$ has a $\chi^2$ distribution.

If you compare this to the formulas (5.1) and (5.2) in the link supra, you will see that the $\chi^2$ distribution in Hosmer-Lemeshow's book uses the above property, which only holds if $m_j$ are all large !

If $J \approx n$ then all $m_j$ are more or less equal to one, which is not ''large'', so if $J \approx n$ the variable $X^2$ defined in (5.1) and (5.2) can not be shown to follow a $\chi^2$ distribution.

The p-values are derived under the property that $X^2$ is $\chi^2$, so if you can not show this $\chi^2$ property, then the p-values derived from it are also 'doubtfull'.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.