I'm doing a matched sample evaluation to see if there is a statistical difference in graduation rates at a college for total sample and sub-groups within the sample after a policy change intended to increase graduates. It seems to me that because it's a dichotomous, nominal depend. var., McNemar is appropriate because it's not normally distributed. Am I on the right track or is a z-test better?

  • $\begingroup$ McNemar's test is for two items recorded for the same subjects. Graduation rates are for different subjects, unless I have misunderstood. It seems that you should be using chi-square tests for different samples, for one characteristic, or possibly a logistic regression if you are evaluating several factors simultaneously. $\endgroup$ – David Smith May 26 '17 at 21:38
  • $\begingroup$ Thanks, @DavidSmith. I don't think I explained my plan well; I'm sorry. What I plan to do is create matched or paired samples of students, using key demographic info, for before and after the policy took effect. Then compare if the graduation rates between those paired samples are statistically different. Does that clarify? In the case I tried to describe, my depend. variable is "graduate" or "not graduate", so I think that lends itself to McNemar. Or is there a way to z-test? $\endgroup$ – ACH May 28 '17 at 3:41
  • $\begingroup$ Thanks. Stepping back to your design, the usual reasons to match cases (graduates) and controls (others) are because obtaining the data is difficult or expensive or time consuming. (Sometimes matches are observed naturally, as with twins.) If the data is easy to obtain I would strongly recommend against matching. $\endgroup$ – David Smith May 28 '17 at 18:39

I recommend against matching.

If you do not match you can use logistic regression or other multivariable methods appropriate for a binary outcome.

If you do decide to use matching, then there are models for logistic regression (and similar methods) that are appropriate for matched outcomes, either 1:1 or k:1, that is, where multiple controls are matched to each case.

Usually matching isn't worth doing unless your data collection is quite expensive, involving, for example, an extended interview process or the collection and analysis of samples of material. The special methods needed for analysis are harder to interpret as well. Don't do it unless it saves you real money.

  • $\begingroup$ The question was not "Do you think I should perform a matched pair study or not?" $\endgroup$ – Alexis Aug 8 '18 at 1:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.