# How to Control Familywise Error Rate in a series of Planned McNemar Comparisons

I have paired binary responses from same subjects under 9 different conditions. I conducted Cochran's Q which indicated that there significantly different pairs exist. I want to follow up but NOT with post-hoc comparisons: I am not interested in the 36 possible comparisons of these 9 conditions.

I have selected a smaller subset of pairwise comparisons to analyze using McNemar's test. I can manually apply Bonferroni correction by simply dividing my chosen alpha by the number of comparisons I'm running.

However, I worry that Bonferroni may be punishing me too much and I'm interested in also applying other corrections; perhaps Benjamini-Hochberg. How would I implement these alternative corrections (preferably in R). Packages like rcompanion correct for all pairwise comparisons, but I want to correct for a smaller subset that I will test.

You can do better than (stepwise) Bonferroni for this problem; see Westfall, P.H., Troendle, J.F. and Pennello, G. (2010). Multiple McNemar Tests. Biometrics 66, 1185–1191.

• Thank you very much, this is great. However, the paper is a bit heavy for me to read and I'm trying to finalize these analyses for a submission deadline. Please share if you have R code to implement this method. Otherwise, I will probably try to implement it myself while waiting for reviews and change things during the revision. – beberuhi Aug 28 '18 at 9:38

Answering my own question: It seems that I can just pick a number of p values from my planned comparisons and run them through the p.adjust function in R, choosing hochberg or another method of adjustment:

p.adjust(p, method = p.adjust.methods, n = length(p))


Quick demo: I use the rcompanion package for McNemar pairwise comparisons.

 MCNEMARRESULTS = pairwiseMcnemar(x=..., g=..., block=..., method="hochberg", test="mcnemar")


This of course corrects for all possible pairwise comparisons. I can load the uncorrected p-values from the above output into a dataframe and feed it into the p.adjust function. I'm novice in R so haven't finalized the code for that, but you get the idea.

I have checked that the p.adjust function with hochberg correction returns the same adjusted p values as the rcompanion package. However, there were some very minor differences.