Effect of sample size on Chi-squared distribution

Question

As the degrees of freedom increase, the chi-square curve approaches a normal distribution. But what about increasing the sample size? If I am using degree of freedom =1, but with different sample sizes (eg n=10, 30 and 50), will the chi-square curve become more normal with increasing sample size?

Answer 1

Assuming that you are asking about the situation in a goodness of fit test or perhaps an independence test where the number of categories is held constant but the total sample size increases:

The chi-squared curve doesn't become more normal; the d.f. is unchanged.

What's changing is how well the cell counts are approximated by a normal distribution. More accurately, what's changing is how well suitably scaled cell counts are approximated by a (typically degenerate) multivariate normal distribution, and hence how well the discrete test statistic $\sum_i\frac{(O_i-E_i)^2}{E_i}$ is approximated by a (continuous) chi squared distribution.

It's this approximation that leads to rules like "expected counts >5" in an attempt to get reasonable accuracy of significance levels across a range of possible situations (different hypothesized $p_i$ in a goodness of fit test). The rule is often 'too strict', but occasionally not really strict enough (at least for my preferences of accuracy of p-values).

Example of improving approximation with sample size:

(You can see a larger image by clicking on the plot, or right-clicking to open the plot in a new tab)

The plot on the left shows the distribution function of a Pearson chi-squared goodness of fit test for three groups with null cell probabilities $0.2, 0.3, 0.5$ for a sample of size $n=15$ (black step function) and the continuous approximation by a chi-squared(2) distribution in red. Here the smallest expected count is 3.

This case was not chosen to show a situation where the approximation is particularly poor; its a 'nice' case. The broad picture looks reasonably close but consider that the exact 5% critical value is 5.756 while the chi-squared approximation gives 5.991, so there is some error, though it's not especially impactful if you do your test at the 5% level in this case, in that a 5% test is mildly conservative.

The approximation error has a somewhat larger impact at the 10% level for this specific case.

The 'exact' cdf is simulated (calculated by sampling from its distribution under $H_0$), but the simulation size is such that the worst-case margin of error in the estimate of the cdf is only a little over a pixel (which occurs near the median; near the 5% critical value it's nearer to half a pixel), so the picture is effectively exact; the differences you see are real, not due to sampling error -- if you did a new simulation of the same size the plot would be indistinguishable from this one.

The plot on the right is similar but for n=50 (smallest expected count is 10); as we see, the chi-squared approximation is considerably better.

Answer 2

One of the best ways to understand statistical concepts is just to simulate the random experiments yourself. Here is some R code which illustrates what happens as you increase the sample size.

simulate = function(n){
  data = rpois(n, lambda = 1)
  Obs.0 = sum(data == 0)
  Obs.1 = sum(data == 1)
  Obs.2 = sum(data >= 2)
  Exp.0 = dpois(0, lambda = 1)*n
  Exp.1 = dpois(1, lambda = 1)*n
  Exp.2 = n - (Exp.0 + Exp.1)
  chi = 0
  chi = chi + (Obs.0-Exp.0)^2/Exp.0
  chi = chi + (Obs.1-Exp.1)^2/Exp.1
  chi = chi + (Obs.2-Exp.2)^2/Exp.2
  chi
}

result = replicate(1e2, simulate(1e2))
plot(density(result))
curve(dchisq(x,2), add=TRUE, col="red")

Let us explain what this code does. When you run simulate(n), the computer generates $n$ (independent) random samples from Poisson(1). We created three categories, whose where the sample = 0, those were sample = 1, and those were sample >= 2. We calculate the expected count vs the actual count. In the end the chi-squared statistic is calculated for that data set.

But this is just 1 number. So now we replicate this process $10^2$ times. Generating $10^2$ different values of the chi-squared statistic. Then we tell the computer to draw the estimate distribution function for all those chi-squared values. Finally, the idealized chi-squared distribution is drawn in red.

Notice, if you increase the replication number from $10^2$ slowly up to $10^5$, the fit gets closer to the idealized chi-squared.