I have paired data from 9 individuals. The response variable is ordinal with levels 1-5. As the sample is very small and the data are categorical, I thought the appropriate test was McNemar's test. However I get McNemar's chi-squared = NaN and therefore a p-value = NA. What should I do to test whether the effect is significant?
My data:
before
4 3 4 5 4 4 4 5 3
after
5 4 3 5 5 4 5 5 4
I wonder if you really want McNemar's test (more specifically the McNemar-Bowker test). These are tests to see if the marginal proportions are the same (see here). Under the assumption that pairings can be recovered from the orders of the two vectors, here is a table of your data with the marginal proportions computed:
bef = c(4, 3, 4, 5, 4, 4, 4, 5, 3)
aft = c(5, 4, 3, 5, 5, 4, 5, 5, 4)
table(bef, aft)
# aft
# bef 3 4 5
# 3 0 2 0
# 4 1 1 3
# 5 0 0 2
table(bef)/sum(table(bef))
# bef
# 3 4 5
# 0.2222222 0.5555556 0.2222222
table(aft)/sum(table(aft))
# aft
# 3 4 5
# 0.1111111 0.3333333 0.5555556
Since you state that your variables are ordered, it seems more appropriate to assess if one set has higher values than the other. That would use the Wilcoxon signed rank test:
wilcox.test(bef, aft, paired=TRUE)
# Wilcoxon signed rank test with continuity correction
#
# data: bef and aft
# V = 3.5, p-value = 0.1294
# alternative hypothesis: true location shift is not equal to 0
Because you have ties, you cannot use the exact version of the test; you use the asymptotic approximation. If that bothers you due to the small sample size, you could simulate the null and compute the p-value that way. There are different ways to do that, but I'm not sure how much difference they will make in practice. Bootstrapping is generally not recommended with small samples. You could try a permutation-based version, or you could do a Monte-Carlo simulation of the null based on the marginal distribution of the values (ignoring timepoint).
It is almost certainly because you have a pair of cells which both have zeroes (3, 5) and (5, 3). If you look at the way the test is defined that will lead to problems.
What to do is a problem. You could always merge categories but that may not make scientific sense. I think you may have to program your own solution. You could define a log-linear model of quasi-symmetry without including the offending cells (treating them as structural zeroes) but that may not be a sensible assumption either. More data would help but I suppose that may not be an option.
You can use the exact binomial test. Let a be the number of cases where before < after. Let b be the number of cases where before > after.
H0: Getting a smaller value after is as likely as getting a bigger value after.
So you can use
bef = c(4, 3, 4, 5, 4, 4, 4, 5, 3)
aft = c(5, 4, 3, 5, 5, 4, 5, 5, 4)
a <- length(bef < aft)
b <- length(bef > aft)
binom.test(a, a + b, p=0.5)
