Which test should I use to assess for the statistical signficance of changes in multiple binomial dependent variables from T1 to T2? Students have completed a test containing 20 questions at both T1 and T2, with an intervention in the interval. Scores for each question are either 0 (incorrect) or 1 (correct). I am interested in knowing whether the improvement in students' scores was significantly greater for some questions than for others. I am thinking that this may involve an extension of the McNemar test, but open to all suggestions. Thanks!
 A: You can use multi-level logistic regression. You've only got one dependent variable, correctness. You have multiple independent variables nested within student. In R you can use lmer to construct the model. It would look something like.
m <- lmer( answer ~ treatment * Q + (treatment * Q | student), family = 'binomial', data = mydata)

That would allow for there to be random effects of question and treatment within student as well as overall correctness variability within student but you would also be able to assess fixed effects of treatment and question. What you seem to really want to know is all of the treatment by question interactions and that model provides them.
In order to analyze all of the questions with any kind of reliability you really should have a lot of students taking the test (hundreds). The general effect of treatment could be assessed with fewer. Also, if you know the categories, the kinds of questions you think differ, then you could replace the individual question variable with that. It would be much more sensible and make this look much less like a fishing expedition.
