1
$\begingroup$

I have paired data.The response variable is categorical with levels 1-5. And I have used both McNemar's test and Wilcoxon rank sign test and I get different p-values. For McNemar's test a p-value of 0.0396; and for Wilcoxon rank sign test a p-value of 0.538.

before=c(4,3, 5, 3, 5, 4, 3, 3, 3, 5, 5, 4, 5, 4, 5, 5, 5, 5, 5, 5, 5, 5)
after=c(4, 4, 5, 5, 4, 3, NA, 5, 3, 4, 5, 3, 4, 4, 5, 5, 5, 4, 4, 5, 4, 5)
$\endgroup$
  • $\begingroup$ What do these 2x22 numbers represent? Are they counts or ratings? Explain your data and your task. $\endgroup$ – ttnphns Jul 28 '16 at 11:58
  • $\begingroup$ @ttnphns re edited $\endgroup$ – Mik meadow Jul 28 '16 at 12:01
  • 1
    $\begingroup$ What question are you asking of these data? Those two tests are very different. Here is a good explanation of the McNemar Test: stats.stackexchange.com/questions/76875/… $\endgroup$ – Ashe Jul 28 '16 at 14:35
2
$\begingroup$

Wilcoxon's signed rank test checks if the values after are systematically higher or lower compared to those before, while the chi-squared symmetry test (aka McNemar's test in the binary case) checks for any difference in distribution, not just a shift.

So, if the true distributions before and after would differ mainly in a shift, then Wilcoxon's signed-rank test would have higher power to detect this difference in a random sample. Otherwise (e.g. if there is a tendency for values before of 3 to become 5 and vice versa), then the symmetry test would have higher power.

The problem here is: you should choose the right test before looking at the p values, simply based on the research question and the type of data you got. So it would be a bit funny to write now:

We were interested in any distributional change, so we performed a chi-squared test of symmetry and reached a significant result at the 5% level.

but actually mean:

I want to provide my readers a significant result, thus we have chosen the chi-squared test".

$\endgroup$
  • 1
    $\begingroup$ (+1) Particularly for the advice to decide these things before running tests and elucidating how the tests look for differences in the paired scores. $\endgroup$ – Ashe Jul 28 '16 at 15:07
2
$\begingroup$

McNemar's is for nominal variables so it's not a proper test in this case. Wilcoxon signed rank is more appropriate.

$\endgroup$
  • 3
    $\begingroup$ Well, technically you are correct but stats.stackexchange.com/questions/65479/… describes the generalisation to three or more categories which some people call Mcnemar-Bowker and other people call Mcnemar $\endgroup$ – mdewey Jul 28 '16 at 13:59
  • 1
    $\begingroup$ @mdewey, thanks. Even with the generalized version it's not appropriate. Both versions of McNemar's examined asymmetry or discordant and jumping from category 1 to 2 or from category 1 to 3 are not considered a different kind of change. Yet, the OP's data are rank data and in fact jumps from rank 1 to 2 and rank 1 to 3 are of different magnitudes and such important information should be included in the test. I've taken out the binary part from the answer to make it better. $\endgroup$ – Penguin_Knight Jul 28 '16 at 14:22
  • 2
    $\begingroup$ How do you know this is the appropriate use? The OP hasn't at all specified what they are testing for. The Wilcoxon signed-rank test and the McNemar test are extremely different in what is tested. $\endgroup$ – Ashe Jul 28 '16 at 14:34
  • 1
    $\begingroup$ @Ashe, I do not know but that they test different things has no bearing in my answer. By the fact that Wilcoxon was used I guessed it's rank data, by the fact that it's rank data I speculated that McNemar's is not a good test. It's an interactive platform and if the OP has question/clarification he/she is welcomed to put that on here. $\endgroup$ – Penguin_Knight Jul 28 '16 at 17:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.