I ask this question because Bowker test (McNemar extension for $k\times k$ tables with $k>2$) requires a large number of observations for fitting well and also requires (as a rule of thumb) that the difference between symmetric cells be $> 5$.

Currently I'm working with $5 \times 5$ tables (pre-post responses to Likert-scale questionnaire items). I would like to know if there were significant changes in the distribution of students answers and if so, verify if the responses were higher or lower in the Likert-scale in comparison of the first stage of the experiment.

Here I have a small number of observations (25 students) What kind of statistic test can I use to test table symmetry that meets my requirements?

  • 2
    $\begingroup$ Will, instead exact (exhausive permutation) test, a Monte Carlo (simulated permutation) test suit you? You could compute the chi-square statistic of the test for each of many permutations and then assess the alpha-level cut-off tail in its distrubution. $\endgroup$
    – ttnphns
    Aug 11, 2015 at 9:45
  • 1
    $\begingroup$ library.wolfram.com/infocenter/MathSource/7634 is what you ask for. $\endgroup$
    – ttnphns
    Aug 11, 2015 at 9:55
  • $\begingroup$ Hi @ttnphns I have already searched that source (Mathematica package) but: 1. I don't have that software, 2. I wanted to understand the algorithm implementation and e.g. coding it in R on my own, 3. The detail of the algorithm is referenced in a paper written in German (I don't speak German). So if you can figure out how they implemented that algorithm let me know please, thanks for your answers! $\endgroup$ Aug 11, 2015 at 15:41
  • $\begingroup$ Unfortunately, I won't be able to help with that algo. $\endgroup$
    – ttnphns
    Aug 11, 2015 at 17:00
  • 1
    $\begingroup$ How is the statistic defined? Is it this one: $Q_B=\sum\sum_{i<j} (n_{ij}-n_{ji})^2/(n_{ij}+n_{ji})$. If so, the paper by Krampe & Kuhnt (2007) "Bowker's test for symmetry and modifications within the algebraic framework", Comp.Stat.&Data Analysis, 51 (9), pp 4124–4142 (tech report version here) may be useful. $\endgroup$
    – Glen_b
    Aug 12, 2015 at 7:12

1 Answer 1


I think this is going to work. Given a contingency table, the distribution of the off-diagonal elements $n_{ij},\,i \neq j$, conditional on the sum of the complementary off-diagonal cells, $N_{ij} = n_{ij} + n_{ji}$, can be written as the product of $K(K - 1)/2$ binominal random variables, that is

$$ P(\mathbf{n}) = \prod_{i < j} \binom{N_{ij}}{n_{ij}} \pi_{ij}^{n_{ij}} ( 1- \pi_{ij})^{n_{ij}},$$

where $\mathbf{n}$ is a vector with elements $n_{ij}$ and $\pi_{ij} = \mathbb{E}(n_{ij}/N_{ij}|N_{ij})$. Under the null hypothesis of complete symmetry, $\pi_{ij} = \pi_{ji} = 1/2$, and thus the permutation distribution is given by

$$ P_{0}(\mathbf{n}) = \prod_{i < j} \binom{N_{ij}}{n_{ij}} \left( \frac{1}{2}\right)^{N_{ij}}.$$

The exact significance test is performed by evaluating

$$ p = \sum_{k \in K} P_{0}(\mathbf{n}),$$

where $K = \{k : P_{0}(\mathbf{n}) < P(\mathbf{n})\}$.

The following R code should do the job.

bowker <- function(mat) {
  n <- dim(mat)[1]
  P_0 <- 1
  P <- 1
  p <- 0
  for (i in 1:(n-1)) {
    for (j in (i+1):n) {
      N <- mat[i,j] + mat[j,i]
      pi <- mat[i,j] / N
      P_0 <- P_0 * choose(N, mat[i,j]) * (1/2)^N
      P <- P * choose(N, mat[i,j]) * pi^mat[i,j] * (1 - pi)^mat[i,j]
      if (P > P_0) {p <- p + P_0}
  return(list(P_0 = P_0, P = P, p = p))

# asymmetric contingency table
mat_asym <- matrix(seq(1,5), ncol = 5, nrow = 5)

# symmetric contingency table
mat_sym <- matrix(2, ncol = 5, nrow = 5)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.