I have a scenario where 3 subjects evaluate 12 cases of a problem with two pieces of equipment. Each subject evaluates each case with equipment A, then equipment B. The outcome variable is binary, 0 (the subject came to an incorrect conclusion) or 1 (the subject came to the correct conclusion)

My hypothesis is that equipment B is better for evaluating the problem in question, which results in the prediction that equipment B will produce more 1s than A does.

Someone suggested the McNemar, but each subject is being tested more than twice (12 cases x 2 tests, not 1 case x 2 tests). Someone else suggested the Cochran Q, but the analysis is producing p values based on differences between the subjects, which may be part of our analysis (the effect of subjects being potentially relevant), but is not related to our research question directly. Another suggested the McNemar-Bowker, but I am not sure that solves the problem either.

At this point I am considering doing a mixed effect regression model. Thoughts?

  • $\begingroup$ See if the U-statistic method of examining all possible relevant pairs of observations pertains: hbiostat.org/bbr/obsvar $\endgroup$ Oct 11, 2023 at 13:59
  • $\begingroup$ Thank you very much. That does seem relevant to my design, and I missed it in my searches. $\endgroup$
    – Abrams
    Oct 11, 2023 at 15:11


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.