Why is chi-square used for contingency tables?

Question

When one is checking whether two categorical variables are independent, it is customary to construct a contingency table and use the chi-square test. However, why can the chi-square distribution with k degrees of freedom (i.e. the sum of squares of k standard normal variables) stand proxy for contingency tables with k degrees of freedom? Is there some kind of proof that they are related? As far as I can see, most statistics textbooks just leave this unexplained.

Answer 1

In a contingency table the null hypothesis states that the variables in the rows and the variable in the columns are independent.

The cell counts $X_{ij}$ are assumed to be poisson distributed with mean $E_{ij}$ and as they are poisson, their variance is also $E_{ij}$.

Asymptotically the Poisson distribution approaches the normal distribution with mean $E_{ij}$ and standard deviation with mean $\sqrt{E_{ij}}$ so, asymptotically $\frac{(X_{ij}-E_{ij})}{\sqrt{E_{ij}}}$ is approximately standard normal.

If you square standard normal variables and sum these squares then the result is a chi-square random variable so

$\sum_{i,j} \left( \frac{(X_{ij}-E_{ij})}{\sqrt{E_{ij}}} \right)^2$

has a (asymptotically) a chi-square distribution. Asymptotics must hold and that is why most textbooks state that the result of the test is valid when all expected cell counts $E_{ij}$ are larger than 5, but that is just a rule of thumb that makes the approximation ''good enough''.

Answer 2

Consider first a binomial; $X$ and $Y=n-X$ are a pair of random variables; once we condition on $n$, knowing either of $X$ or $Y$ tells you the other. If we consider the distribution of $(X,Y)$ for large $n$ we can write a (degenerate) "bivariate" normal approximation for it (in that the lies on a one-dimensional subspace $X+Y=n$), by specifying the means, variances and covariances in terms of $n$ and the success probability $\pi$ ($\mu=(n\pi,n(1-\pi))^\prime$, $\text{Var}(X)=\text{Var}(Y)=n\pi(1-\pi)$ and $\text{Cov}(X,Y)=-n\pi(1-\pi)$).

If you write the ordinary normal approximation to the binomial out in this case it's very easy to see that $Z^2$ in the Z-test is equivalent to the chi-squared statistic. (Indeed the chi-squared statistics $Z^2$ corresponds to (minus twice) the exponent in the normal density; this is common to the various statistics mentioned.)

Similarly with a multinomial, $X_1,X_2,....,X_k$ such that $\sum_i X_i =n$. Again we can write down the means, variances and covariances between each of the terms, and while we have $k$ variates, they sit on a $k-1$ dimensional subset.

Similarly consider a $2\times 2$ contingency table. If we consider it as a pair of binomials $(X_1,X_2)$ (which under the null have a common $\pi$) then we can most easily work with $X_1/n_1-X_2/n_2$ (as if we were doing the Z test for two-proportions) and generate the normal approximation for that; the usual chi-square statistic will be equivalent to the exponent in the normal (it takes some algebra to show this, though). However, more akin to the way we treat the 2x2 chis square, we could write our variate as two sets of pairs $X_1,Y_1,X_2,Y_2$ and approximate that by a multivariate normal (though the four-variates live again on a subspace of dimension 2). Once we estimate that common $\pi$ parameter, however, we lose an additional degree of freedom and it effectively becomes univariate again.

Again we can extend this up to multinomial variates (and a $2\times k$ table) and then to larger tables.

In each case, the exponent in the approximating normal density can be written in the form of a chi-squared statistic which may be shown to have an asympotic normal approximation.

For example the $2\times k$ multinomial case is discussed in Lehmann's Elements of Large-Sample Theory Sec 5.5, Example 5.5.5 "The multinomial two-sample problem." p329 (while the statistic he shows doesn't look at first glance like the chi-square statistic -- he writes it in terms of weighted sums of squares of differences in proportions -- it can be shown to be equivalent. If you check that out, note that his $k$ is one smaller than mine here throughout. By dealing directly with the differences of the proportions in the within-category pairs he avoids a deal of effort, but this doesn't generalize to the more-than-two-sample case.)

[Alternatively we can condition on the other dimension from the beginning, whence the (1,1) cell is hypergeometric, and we can apply a normal approximation to that (the other cells are all in turn hypergeometric but fixed given the first one and the margins), and so on up for larger tables.]