Consider the following problem:
You have 3 groups. Your null hypothesis is that the 3 groups are distributed uniformly. Your data consists of 90 observations. Under the null hypothesis we expect to observe 30 outcomes in each of the 3 groups. The actual data consists of the split 40/30/20. We would like to test our hypothesis and make a decision if we need to reject. The usual way is to approach this problem by calculating the chi-squared statistics: $$ \chi^2 = \tfrac{(40-30)^2}{30} + \tfrac{(30-30)^2}{30} + \tfrac{(20-30)^2}{30} \approx 6.7 $$ Now we can calculate the $p$-value by looking this value up:
1 - pchisq(6.7, df = 2)
Therefore, the $p$-value is equal to approximately, 0.035.
There is an alternative method to solve this problem that is based on the principle that, "just calculate the mean and variance, and then estimate it by a normal". To be more specific, let $X$ be the random vector that describes the distribution of the 3 groups. Since our null hypothesis is a uniform distribution among the 3 groups we can say that, $$ X \sim \text{Multinomial}(90, (\tfrac{1}{3},\tfrac{1}{3},\tfrac{1}{3}))$$ The mean of this random vector is equal to $(30,30,30)$. The (co)variance of $X$ is equal to, $$ A = \begin{bmatrix} +20 & -10 & -10 \\ -10 & +20 & -10 \\ -10 & -10 & +20 \end{bmatrix} $$ (since the variance on the diagonal is $90\times \tfrac{1}{3}\times \tfrac{2}{3}$ and the covariance between the components is $-90\times \tfrac{1}{3}\times \tfrac{1}{3}$).
Thus, we can estimate the (distribution of) $X$ by a (multivariable) normal, $$ X \sim \text{Normal}\left( (30,30,30), \Sigma \right) $$ The distance between the observed data $(40,30,20)$ and the expected data $(30,30,30)$ equates to $\sqrt{200}$. We can now calculate the $p$-value by finding how rare is it for this normal to exceed a distance of greater than $\sqrt{200}$ from its mean?
library(MASS)
count = 0
for(n in 1:1e5){
sample = mvrnorm( 1, c(30,30,30), A )
if( sqrt( sum((sample - c(30,30,30))^2) ) < sqrt(200) ){
count = count + 1
}
}
1 - count/1e5
The above simulation produces a $p$-value of 0.036.
My question is what is the real benefit that we gain from using the $\chi$-squared statistic? The normal approximation approach is very intuitive and conceptually clearer.
The most obvious reply would be, "because the $\chi$-squared approach is easier to implement". That is only true because there is a standard $\chi$-squared calculator. We can, of course, develop our own calculator that uses this normal approximation instead.
Therefore, what would be a real benefit to using the $\chi$-squared? For instance, are there situations were the answers would be radically different?
