The $\chi^2$ statistic has been frequently employed in the hypothesis testing of the multinomial distributions. But after looking at the derivation of such a process (where they use Taylor expansion and discard all the remaining terms after the power of 3), I become suspicious and wonder if there's any counterexample to the statement (in which a contradiction arises after directly applying $\chi^2$ statistic to the multinomial distribution). Is there any way to strictly prove the validity of such an application of $\chi^{2}$ statistic (with mathematical rigor), or is there any counterexample to the argument?


1 Answer 1


In statistical genetics, "empirical p-value testing" is often used to get around distributional assumptions. For example, consider a multiway table with $r$ rows and $c$ columns. For each row, the multinomial probability of the row total $\{ n_{i.} \}$ is

\begin{equation} P(\{n_{i.}\})= \frac{n_{i.}!}{\prod_j n_{ij}!} \prod_j \pi_{j|i}^{n_{ij}}, \end{equation}

where $\pi_{j|i}$ is the conditional probability in column $j$ ($j=1,2,...,c$) given row $i$.

The probability of the table $\{n_{ij}\}$ is the product of all of the multinomial row probabilities

\begin{equation} P(\{n_{ij}\})= \prod_i \left( \frac{n_{i.}!}{\prod_j n_{ij}!} \prod_j \pi_{j|i}^{n_{ij}}\right). \end{equation}

Under independence, we assume that in each column the conditional cell probabilities are equal (including the row total), shown as $\pi_{j|1}=\pi_{j|2}= \cdots = \pi_{j|r}=\pi_{.j}$. Thus, let's substitute $\pi_{.j}$ for $\pi_{j|i}$ and $n_{.j}$ for $n_{ij}$ in the above equation and obtain \begin{equation} \begin{split} P(\{n_{ij}\})&= \prod_i \left( \frac{n_{i.}!}{\prod_j n_{ij}!} \prod_j \pi_{.j}^{n_{.j}} \right)\\ &= \frac{\prod_i n_{i.}! \prod_j \pi_{.j}^{n_{.j}} } {\prod_i \prod_j n_{ij}!}. \\ \end{split} \end{equation}

The distribution of the cell counts in each row, however, also depend also on $\pi_{.j}$. Fisher assumed these to be sufficient nuisance parameters, and conditioned on them so that the resulting conditional probability does not depend on them. The contribution of $\pi_{.j}$ to the product multinomial distribution depends on the data through $n_{.j}$, which are the sufficient statistics. Each column total $n_{.j}$ has the multinomial distribution \begin{equation} P(\{n_{.j}\})= \frac{n!}{\prod_j n_{.j}!} \prod_j \pi_{.j}^{n_{.j}}. \end{equation}

The probability of $\{n_{ij}\}$ conditional on $\{n_{.j}\}$ equals the probability of $\{n_{ij}\}$ divided by the probability of $\{n_{.j}\}$, so we invert the above ratio and multiply as \begin{equation} \begin{split} P(\{n_{ij}\}|\{n_{.j}\}) &= \left( \frac{\prod_i n_{i.}! \prod_j \pi_{.j}^{n_{.j}} } {\prod_i \prod_j n_{ij}!} \right) \left( \frac{\prod_j n_{.j}!}{ \prod_j \pi_{.j}^{n_{.j}}{n!}} \right)\\ &= \frac{\prod_i n_{i.}! \prod_j n_{.j}} {n!\prod_i \prod_j n_{ij}!}. \\ \end{split} \end{equation}

$P$ above is first calculated from the observed data. Permutations are then used by re-shuffling categories for one of the two categorical variables being considered, each time recalculating $P^{(b)}$ $(b=1,2,...,B)$ repeatedly in order to simulate the null distribution. After $B$ iterations, the exact p-value is \begin{equation} P=\frac{\# \{b: P^{(b)}< P\}}{B} . \end{equation}

Randomization Test for $\chi^2$. You will notice that even for small tables, e.g. 3x4, 4x4, 4x5, 5x5, the number of iterations required for fully implementing the exact form of the test can exceed than $10^4$ configurations. Therefore, one can use the following randomization test:

  1. Calculate the raw chi-squared value for the table, call this $\chi^2$.

  2. Randomly shuffle (permute) one of the grouping variable's values.

  3. Calculate chi-squared using the grouping variables (with one of them shuffled), call this $\chi^2_{(b)}$.

  4. Repeat steps 2-3 $B=10,000$ times (default).

The p-value for this randomization test is

\begin{equation} P=\frac{\#\{b: \chi^2_{(b)} \geq \chi^2\}}{B}, \end{equation}

that is, the number of times the chi-squared value for the permuted configurations exceeds or is equal to the chi-squared value for the raw data configuration (table), divided by the number of iterations.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.