Chi-Square Difference Test for Nested Models with a Continuous Outcome

Question

Apologies if this is a very basic question - I'm still definitely a beginner when it comes to statistical tests. I understand that chi-square tests are conducted to determine the relationship between two categorical variables or to determine how well a categorical outcome fits a discrete probability distribution.

However, I often come across articles using a chi-square difference test to compare the fit of nested models, even when the outcome is continuous. I'm struggling to understand this a little - how can we calculate a chi-square value for two models to then test if the difference between them is significant if the DV/outcome is continuous?

Answer 1

Chi-square is a distribution, not a test. Chi-square tests are used for things that are chi-square distributed. There are two different things they are used for:

1. The chi-square test of a contingency table (which is what you're thinking of, when you say they are to determine the relationship between two categorical variables).
2. Likelihood ratio tests are often also called chi-square tests, because the difference in (-2 *) loglikelihood of two models is chi-square distributed. If I fit a model, and then I add a parameter, I might ask if that parameter significantly improved the models. I can compare the likelihood ratios of the two models because this difference is chi-square distributed.

(Although the first is actually a type of the second.)

Answer 2

The term 'chi-squared test' tends to be applied to almost any test that has a test statistic whose distribution under $H_0$ is (at least approximately) distributed as chi-squared. Something similar happens with the term 'z-test'.

There are perhaps dozens of different tests with this property.

The chi-squared distribution occurs when you sum squares of independent standard normal random variables. More generally (to get mildly technical - while dodging still further technicalities), the chi-squared distribution arises from the quadratic term in the exponent of a multivariate normal. That is, if $\underline{X}\sim N_p(\underline{\mu},\Sigma)$ then $(\underline{x}-\underline{\mu})^\top \Sigma^{-1}(\underline{x}-\underline{\mu})$ has a $\chi^2_p$ distribution, and if there are linear restrictions on the $\underline{X}$ the chi-squared distribution will lose 1 degree of freedom for each such restriction.

The chi-squared tests you mention (goodness of fit and test of independence) are based on a (degenerate) multivariate normal approximation to a multinomial distribution of counts, given marginal total(s), in just this sense.

There are a variety of other tests that use chi-squared distributions for the test statistic under $H_0$, many involving continuous models. Here's a few examples of ways that chi-squared tests may arise:

• the distribution of sample variance under simple random sampling of a normal population is a (scaled) chi-squared. So a one-sample variance test may be a chi-squared test.

• the distribution of sums of squares of residuals from a (correct) Gaussian model with known $\sigma$ is a multiple of a chi-squared.

• the distribution of minus twice the log of the likelihood ratio in a likelihood ratio test is asymptotically chi-squared (Wilks' theorem) -- indeed there are a number of tests of nested models that result in chi-squared distributions (for some models these will be exact, for others asymptotic); this includes Wald tests and Lagrange multiplier (/score) tests.

The case you mention doesn't specify whether we're dealing with an assumption of normality or not. It might be an 'exact' small sample test or it might be an asymptotic test.