# Does $\chi^{2}$ statistic always works for multinomial hypothesis testing?

The $$\chi^2$$ statistic has been frequently employed in the hypothesis testing of the multinomial distributions. But after looking at the derivation of such a process (where they use Taylor expansion and discard all the remaining terms after the power of 3), I become suspicious and wonder if there's any counterexample to the statement (in which a contradiction arises after directly applying $$\chi^2$$ statistic to the multinomial distribution). Is there any way to strictly prove the validity of such an application of $$\chi^{2}$$ statistic (with mathematical rigor), or is there any counterexample to the argument?

In statistical genetics, "empirical p-value testing" is often used to get around distributional assumptions. For example, consider a multiway table with $$r$$ rows and $$c$$ columns. For each row, the multinomial probability of the row total $$\{ n_{i.} \}$$ is

$$$$P(\{n_{i.}\})= \frac{n_{i.}!}{\prod_j n_{ij}!} \prod_j \pi_{j|i}^{n_{ij}},$$$$

where $$\pi_{j|i}$$ is the conditional probability in column $$j$$ ($$j=1,2,...,c$$) given row $$i$$.

The probability of the table $$\{n_{ij}\}$$ is the product of all of the multinomial row probabilities

$$$$P(\{n_{ij}\})= \prod_i \left( \frac{n_{i.}!}{\prod_j n_{ij}!} \prod_j \pi_{j|i}^{n_{ij}}\right).$$$$

Under independence, we assume that in each column the conditional cell probabilities are equal (including the row total), shown as $$\pi_{j|1}=\pi_{j|2}= \cdots = \pi_{j|r}=\pi_{.j}$$. Thus, let's substitute $$\pi_{.j}$$ for $$\pi_{j|i}$$ and $$n_{.j}$$ for $$n_{ij}$$ in the above equation and obtain $$$$\begin{split} P(\{n_{ij}\})&= \prod_i \left( \frac{n_{i.}!}{\prod_j n_{ij}!} \prod_j \pi_{.j}^{n_{.j}} \right)\\ &= \frac{\prod_i n_{i.}! \prod_j \pi_{.j}^{n_{.j}} } {\prod_i \prod_j n_{ij}!}. \\ \end{split}$$$$

The distribution of the cell counts in each row, however, also depend also on $$\pi_{.j}$$. Fisher assumed these to be sufficient nuisance parameters, and conditioned on them so that the resulting conditional probability does not depend on them. The contribution of $$\pi_{.j}$$ to the product multinomial distribution depends on the data through $$n_{.j}$$, which are the sufficient statistics. Each column total $$n_{.j}$$ has the multinomial distribution $$$$P(\{n_{.j}\})= \frac{n!}{\prod_j n_{.j}!} \prod_j \pi_{.j}^{n_{.j}}.$$$$

The probability of $$\{n_{ij}\}$$ conditional on $$\{n_{.j}\}$$ equals the probability of $$\{n_{ij}\}$$ divided by the probability of $$\{n_{.j}\}$$, so we invert the above ratio and multiply as $$$$\begin{split} P(\{n_{ij}\}|\{n_{.j}\}) &= \left( \frac{\prod_i n_{i.}! \prod_j \pi_{.j}^{n_{.j}} } {\prod_i \prod_j n_{ij}!} \right) \left( \frac{\prod_j n_{.j}!}{ \prod_j \pi_{.j}^{n_{.j}}{n!}} \right)\\ &= \frac{\prod_i n_{i.}! \prod_j n_{.j}} {n!\prod_i \prod_j n_{ij}!}. \\ \end{split}$$$$

$$P$$ above is first calculated from the observed data. Permutations are then used by re-shuffling categories for one of the two categorical variables being considered, each time recalculating $$P^{(b)}$$ $$(b=1,2,...,B)$$ repeatedly in order to simulate the null distribution. After $$B$$ iterations, the exact p-value is $$$$P=\frac{\# \{b: P^{(b)}< P\}}{B} .$$$$

Randomization Test for $$\chi^2$$. You will notice that even for small tables, e.g. 3x4, 4x4, 4x5, 5x5, the number of iterations required for fully implementing the exact form of the test can exceed than $$10^4$$ configurations. Therefore, one can use the following randomization test:

1. Calculate the raw chi-squared value for the table, call this $$\chi^2$$.

2. Randomly shuffle (permute) one of the grouping variable's values.

3. Calculate chi-squared using the grouping variables (with one of them shuffled), call this $$\chi^2_{(b)}$$.

4. Repeat steps 2-3 $$B=10,000$$ times (default).

The p-value for this randomization test is

$$$$P=\frac{\#\{b: \chi^2_{(b)} \geq \chi^2\}}{B},$$$$

that is, the number of times the chi-squared value for the permuted configurations exceeds or is equal to the chi-squared value for the raw data configuration (table), divided by the number of iterations.