I'm designing a study where I have a control group and a treatment group and measure rates of completion (0 if complete, 1 if not complete) of the different groups (between subjects design). I consider the treatment group successful if a N-1 two proportions test shows a statistically significant difference.

However, I also want to know if there is a statistically significant difference between two different recruitment methods. My thought was to take the difference between the rates of completion in the treatment group and the control group and compare that between the two different recruitment methods (e.g., method #1 produces a boost of +3 percentage points in the treatment group and method #2 produces a boost of +6 percentage points in the treatment group) with a N-1 two proportions test. Is that correct?

Additionally, what equation could I use to calculate the necessary sample size for achieving statistical significance with various assumptions about the difference in percentage point increases (assuming a desired power of 80% and an alpha of 0.05)?

Update/conclusion: we ended up deciding to go with a factorial logistic regression.

  • $\begingroup$ You're going to need to give us more information. What is the study about? How do the "recruitment methods" related to what's going on? Etc. $\endgroup$ – gung Sep 25 '16 at 0:09
  • $\begingroup$ @gung Sure. We're a program that recruits participants into a program that aims to increase vaccination rates. We're recruiting from referrals and from social media, but we're worried that people recruited over social media will not be benefited by the program nearly as much as people recruited through referrals. $\endgroup$ – peterhurford Sep 25 '16 at 0:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.