I adopt a within-group experiment, such that the same participant uses 3 devices to perform a task, which has a result "pass" or fail. My data looks like this:

        Pass  Fail
deviceA   21    13
deviceB   15    20
deviceC   9     25

My questions are:

  1. Is McNemar's test appropriate for this data?
  2. In R, when I run mcnemar.test(data), I got the following error:

    "'x' must be square with at least two rows and columns"

How can I run this $3 \times 2$ data for R's mcnemar.test?

  • 1
    $\begingroup$ I have the same general question. According to Wikipedia, McNemar's test should be applied to 2*2 test. Not an expert here, but you might be able to test in pairs: deviceA vs deviceB, deviceB vs deviceC, deviceA vs deviceC. I am not sure if this would be equivalent though but maybe somebody reading this could help $\endgroup$ – toto_tico Jan 19 '14 at 5:45
  • 1
    $\begingroup$ McNemar test is only for square frequency tables with rows/columns defined by the two variables with the same categories. Classic McNemar is for 2x2 table; its extension McNemar-Bowker is for kxk table. Your table is not appropriate for McNemar. What is your hypothesis, what do you want to test? $\endgroup$ – ttnphns Jan 19 '14 at 6:52
  • $\begingroup$ @ttnphns my null hypothesis is that the device type doesn't affect participants' task performance (pass or fail). Since McNemar and its extension both requires the data matrix to be square, then I assume McNemar is not appropriate for my data? In that case, should I test each pair of device, for example, deviceA and deviceB, using McNemar? Thanks! $\endgroup$ – Ida Jan 19 '14 at 7:35

So, you have a 3-level factor "device", which is a repeated-measures factor. The response data are binary: 1 (pass) and 0 (fail).

respondent  deviceA  deviceB  deviceC
1              0        1        0
2              1        1        0
3              1        1        0
4              0        0        1

You want to test if the factor affects the result, i.e. if the 3 devices differ significantly. Null hypothesis: no differences between the 3 devices in the population; alternative hypothesis: there is some difference, at least between some 2 of the 3 devices.

Use Friedman's test (= Friedman's nonparametric "analysis-of-variance"). When the data values are all binary, this test is then also known as Cochran's Q test.

Please note that the frequency table corresponding to your analysis is size 2x2x2, not 2x2 as McNemar's test implies (or kxk, for McNemar-Bowker), nor 3x2 as you thought. Cochran's Q test is the extension of McNemar's test from 2-way table to multi-way table.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.