Chi Squared Test

Question

In the Chi Squared Test we build up a statistic $Q$ which converges in law to a $\chi^2$ as the number $n$ of observations goes to infinity. So, if $n$ is "big enough", we choose to approximate $Q$ with the $\chi^2$.

I don't understand this approximation. How can we deduce information on $Q$ by knowing its limiting law? The limit does depend only on an arbitrary tail of the sequence, hence doesn't depend on $Q$.

Answer 1

Goodness-of-fit statistic. Suppose you want to test whether a die is fair by rolling it 600 times. Then you would expect, on average, to see each face $E = 100$ times. If the observed counts for faces $i = 1, \dots, 6$ are $X_i,$ then the chi-squared statistic is

$$Q = \sum_{i=1}^6 \frac{(X_i - E)^2}{E} \stackrel{aprx}{\sim} \mathsf{Chisq}(\nu = 6-1=5),$$ the chi-squared distribution with 5 degrees of freedom.

Test at the 5% level. Then we would reject the null hypothesis that the die is fair at the 5% level of significance, if $Q \ge t_c = 11.07,$ where the critical value $q_c$ cuts 5% of the probability from the upper tail of $\mathsf{Chisq}(5).$

qchisq(.95, 5)
[1] 11.0705

Experience has shown that the approximation is reasonably good in such circumstances provided that $E > 5,$ which is true in our case.

Illustration by simulation. A simulation in R of this situation with a fair die is as below. Because we are simulating rolls of a fair die, we expect to reject in about 5% of the 100,000 iterations. The simulated rejection rate is indeed very nearly 5%.

set.seed(709)  # for reproducibility
m = 10^5       # iterations of the 600-roll experiment
q = replicate( m,  
      sum((tabulate(sample(1:6, 600, rep=T))-100)^2/100) )
mean(q > 11.0705)
[1] 0.04974

A histogram of the simulated distribution of $Q$ is a reasonably good fit to the density function of $\mathsf{Chisq}(5).$

hist(q, prob=T, br=40, col="skyblue2")
curve(dchisq(x, 5), add=T, n=1001, col="red", lwd=2)

The statistic $Q$ is discrete because values change by small increments as the counts change at random. However, the continuous chi-squared distribution turns out to be a very good approximation to the distribution of $Q$ in the circumstances illustrated.

Power of the test for a biased die. By contrast, if we simulate using a die that is somewhat biased against showing $1$'s (in favor of $6$'s), then we see that the goodness-of-fit test is very likely to reject the null hypothesis that the die is fair. The power of the test is about 97%.

set.seed(1776)         # for reproducibility
m = 10^5               # iterations of the 600-roll experiment
p = c(2,3,3,3,3,4)/18  # probabilities for biased die
q = replicate( m,  
      sum((tabulate(sample(1:6, 600, rep=T, prob=p))-100)^2/100) )
mean(q > 11.0705)
[1] 0.97434

Note: Under the null hypothesis that the die is biased with probabilities $p = (2,3,3,3,3,4)/18,$ the statistic $Q$ has the non-central chi-squared distribution with $\nu = 5$ degrees of freedom and 'noncentrality parameter' $\lambda = n\sum_i (p_i - 1/6)^2/(1/6) = 22.22,$ so that the power of the goodness-of-fit test can be computed in R (without simulation) as $0.971.$

1-pchisq(11.0705, 5, 22.22)
[1] 0.9709646

Answer 2

In general, the exact distribution of a given statistic would be "difficult" to derive. However, thanks to the various convergence theorems (in particular, any such theorem with Gaussian or Wiener limit goes by the name "central limit theorem" [in probability theory]), one can obtain relatively easily the limiting distribution of the statistic (up to some transformation if necessary) that is better known in the sense that it has already been tabulated. One can then compare the realization of the statistic (at the given sample) to the theoretical values in terms of the better-known limiting distribution. In this sense one speaks of approximation in the context of statistics. Simulation studies can easily show how large a sample size is enough, although there exist some rules of thumb for "nice" cases. By the way, please note that the phrase "as $n$ goes beyond every bound" (and any of its equivalences) is merely mathematical, corresponding to the definition of convergence; a convergence theorem alone does not provide further information on the "sufficient" sample size.

As an instance, we know that a $t$ statistic constructed from a normal random sample is to have a $t$ distribution. When the normal assumption is found too unrealistic for the given sample (this could be the case in observational [as opposed to experimental] studies, for instance), still using a $t$ distribution for inference purposes is not necessarily reasonable; in this case, since a $t$ statistic (under weaker conditions) converges weakly or in distribution to the standard normal, we can instead make a statistical decision based on the behavior of standard normal, which is well tabulated.

Regarding the independence of the limiting distribution to the statistic of interest, such a statistic is called asymptotically nonparametric for the intuitive reason.