There are many different Pearson Chi-Square tests. The test statistics are always a sum of scaled and squared deviations. Some are written as a single sum, and some are written as a double sum (the elements of a double sum are usually displayed in a contingency table).

I understand the derivation of the null distribution for some tests, but I don't how these proofs also apply to the other tests. I'm interested in proving the asymptotic distribution for double sum test statistic, the test statistic that tests whether two categorical factors are independent. I'm familiar with proofs for the single sum case, but I cannot connect the two.

My favorite proof is the first one in this paper. It goes as follows:

Suppose we have many $k$-dimensional count vectors, and that they each follow a Multinomial distribution (or Categorical distribution in this case): $\mathbf{X}_1, \mathbf{X}_2, \ldots, \overset{\text{iid}}{\sim} \text{Multinomial}(1,\mathbf{p}).$ The standard central limit theorem gives us $$ \sqrt{n}(\bar{\mathbf{X}} - \mathbf{p}) \overset{d}{\to} \text{Normal}(0, \Sigma) $$ where $n$ is the number of vectors we have and $$ \Sigma = \begin{bmatrix} p_1(1-p_1) & -p_1p_2 & \cdots & -p_1p_k \\ -p_1p_2 & p_2(1-p_2) & \cdots & \vdots \\ \vdots & \vdots & \ddots & \vdots \\ -p_kp_1 & -p_kp_2 & \cdots & p_k(1-p_k) \end{bmatrix} $$ is a (singular/non invertible) covariance matrix. If you remove the last, say, element of $\bar{X}$ (call it $\bar{X}^*$), then you still have $$ \sqrt{n}(\bar{X}^* - \mathbf{p}^*) \overset{d}{\to} \text{Normal}(0, \Sigma^*) $$ where $\Sigma^*$ is now intvertible and so $$ n(\bar{X}^* - \mathbf{p}^*)^\intercal (\Sigma^*)^{-1}(\bar{X}^* - \mathbf{p}^*) \overset{d}{\to} \chi^2(\text{rank}(\Sigma^*)) $$ with $\text{rank}(\Sigma^*) = k-1$.

As the paper explains in equations (7-9) $n(\bar{X}^* - \mathbf{p}^*)^\intercal (\Sigma^*)^{-1}(\bar{X}^* - \mathbf{p}^*)$ is the same as writing the test statistic in the more typical notation: $$ \sum_{i=1}^n \frac{(O_i - E_i)^2}{E_i}. $$

Great, so this is fine, but how do we use the above proof to show that this test statistic $$ \sum_{i=1}^r \sum_{j=1}^c \frac{(O_{ij} - E_{ij})^2}{E_{ij}}\overset{d}{\to} \chi^2( (r-1)(c-1)) $$ follows a chi-square, too? How can we string out or vec-out a contingency table in the right way so that it follows a Multinomial and the same proof applies?

My attempt involves $\text{vec}$-ing out the contingency table, writing that as a multinomial with a probability vector $\mathbf{p} = \text{vec}(\mathbf{a} \mathbf{b}^\intercal)$, but after invoking the central limit theorem and getting a singular normal distribution with covariance $\text{diag}(\mathbf{p}) - \mathbf{p}\mathbf{p}^\intercal$, I'm not sure how to throw out $(r + c - 1)$ elements of the vector to get a nonsingular normal distribution that squares to the test statistic.

  • 3
    $\begingroup$ The real difficulty of generalizing the proof of the goodness of fit test (where $p^*$ is non-random) to the independence test does not lie in from 1D to 2D, instead, it is the estimated parameters (that is, in your last expression, $E_{ij}$ is random, as opposed to $np_i$ which is non-random), which determines the degrees of freedom of the limiting distribution and requires new asymptotic tools. For this reason, the derivation of this very well-known result is rarely detailed in most textbooks. Luckily, Chapter 17 of Asymptotic Statistics by A. W. van der Vaart discussed this in-depth. $\endgroup$
    – Zhanxiong
    Feb 5 at 5:21
  • $\begingroup$ Good reference, yeah 17.3 and 17.4 are what I need $\endgroup$
    – Taylor
    Feb 5 at 19:01


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy