0
$\begingroup$

I have paired binary responses from same subjects under 9 different conditions. I conducted Cochran's Q which indicated that there significantly different pairs exist. I want to follow up but NOT with post-hoc comparisons: I am not interested in the 36 possible comparisons of these 9 conditions.

I have selected a smaller subset of pairwise comparisons to analyze using McNemar's test. I can manually apply Bonferroni correction by simply dividing my chosen alpha by the number of comparisons I'm running.

However, I worry that Bonferroni may be punishing me too much and I'm interested in also applying other corrections; perhaps Benjamini-Hochberg. How would I implement these alternative corrections (preferably in R). Packages like rcompanion correct for all pairwise comparisons, but I want to correct for a smaller subset that I will test.

$\endgroup$
1
$\begingroup$

You can do better than (stepwise) Bonferroni for this problem; see Westfall, P.H., Troendle, J.F. and Pennello, G. (2010). Multiple McNemar Tests. Biometrics 66, 1185–1191.

$\endgroup$
  • $\begingroup$ Thank you very much, this is great. However, the paper is a bit heavy for me to read and I'm trying to finalize these analyses for a submission deadline. Please share if you have R code to implement this method. Otherwise, I will probably try to implement it myself while waiting for reviews and change things during the revision. $\endgroup$ – beberuhi Aug 28 '18 at 9:38
0
$\begingroup$

Answering my own question: It seems that I can just pick a number of p values from my planned comparisons and run them through the p.adjust function in R, choosing hochberg or another method of adjustment:

p.adjust(p, method = p.adjust.methods, n = length(p))

Quick demo: I use the rcompanion package for McNemar pairwise comparisons.

 MCNEMARRESULTS = pairwiseMcnemar(x=..., g=..., block=..., method="hochberg", test="mcnemar")

This of course corrects for all possible pairwise comparisons. I can load the uncorrected p-values from the above output into a dataframe and feed it into the p.adjust function. I'm novice in R so haven't finalized the code for that, but you get the idea.

I have checked that the p.adjust function with hochberg correction returns the same adjusted p values as the rcompanion package. However, there were some very minor differences.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.