# McNemar test for this application?

I'm doing a study on the compliance of schools with questions in a standard 12 question questionnaire at two points in time (2010 and 2015). Basically the American Heart Association puts out a suggested list of 12 questions for heart screening and then schools pick and choose which questions they use.

For each question I have data on whether the school included a particular question in their own survey in 2010 and in 2015. It's a simple "yes" or "no" as to whether or not they have the question.

Am I correct that this represents a categorical dependent variable with dependent/"matched" groups as independent variables and therefore mcnemar's test is indicated?

This is an older question, but I'll attempt an answer.

This situation is not a good candidate for McNemar's test.

For McNemar's test, the categories have to be mutually exclusive. That is, in this example, if one school choses Question 1, they could not also choose Question 2, or 3, and so on.

To set up a McNemar's test, you should be able to construct a contingency table where the labels for the columns are the same as the labels for rows. Note also that it's necessary to be able to identify the "before" response for each subject (person, experimental unit) and identify the "after" response for the same subject.

In the following table, note that the column categories and row categories are the same. Also note that the table sums to 34 counts, which is the number of subjects. But note that in a sense there are 68 observations, since each of the 34 subjects had to have a response recorded twice.

         After
Before    Yes  No
Yes      12    7
No        5   10


Since the response in the example in the question is a dichotomous yes/no, the example could probably be analyzed with logistic regression. The right hand side of the model would probably include Question and Time, and probably School as a random effect to take into account the repeated measures nature of the data.

A post-hoc analysis (for example with E.M. means) could be used to determine the differences among questions.

A partial set of sample data follows, with a potential model, written in R notation. Another model may be more appropriate, depending on the nature of the problem and data.

Question  School  Time    Response
1         a       Before  Yes
1         a       After   Yes
1         b       Before  No
1         b       After   Yes
.
.
2
2
.
.

glmer(Response ~ Question + Time + (1 | School), data = Data, family = binomial)