3
$\begingroup$

I would like to calculate a sample size for the 2 sample proportion test when the significance level is assumed (0.05), power is assumed (0.90) and I'd like 2 sample proportion test to detect that there is at least 5% difference in proportions in 2 groups (in favour in 1 specific group).

Any idea on how to calcuate this? Or any reference to read? I've found package in R -> pwr and a function pwr.2p.test that enable to calculate a sample with given sig.level, power and given effect but the effect size shoud be delivered as Cohen's d that I can't connect with my desired effect size = at least 5%.

Learn about this from here http://www.statmethods.net/stats/power.html

Thanks for any advce.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

The sample size depends on how rare the event is and how you split your sample into treatment and control. For example, here's the graphical output of the following Stata power calculation command:

power twoprop .1(.1).5, power(0.9) alpha(0.05) diff(.05) nratio(.5 1) onesided graph

The red line shows that if the control group proportion was $p_1=0.5$ (the binomial event of interest is pretty common), then you would need a total sample size of 3,414 observations to detect a 0.05 percentage point difference favoring the treatment group (assuming equal experimental group sizes $\frac{N_2}{N_1}=1$). If the event was relatively rare, say $p_1=0.1$, you would need 1,496 observations split into two to detect 0.05 pp difference between treatment and control.

If you want to split your groups unevenly in an unbalanced design, say $\frac{N_2}{N_1}=\frac{1}{2}$, then you would need a somewhat larger sample size. The blue line that corresponds to the uneven allocation is higher than the red, even-split line. For $p_1=0.5$, we would need $$N'=N_1' + N_2'= 3,842 + 2,561 + 1,281$$ instead of $$N=N_1 + N_2 = 3,414 = 1,707 + 1,707 $$ You essentially need 2 control people for every treatment person you give up.

enter image description here

The formulas used by Stata can be found here.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks! That's all I was looking for. Can I have last question, just to be clear? So if total sample size is 1,496 that means I need to have 748 in experimental group and 748 in control group if the groups are equal? That's the point of viewing total sample size? Does one perform relevant non-equal-size proportion test? $\endgroup$
    – Marcin
    Nov 8, 2014 at 19:55
  • $\begingroup$ I've just computed similar calculations in R with pwr package and received such results pwr.2p.test( h = 0.151897721390859168, sig.level = 0.05, power = 0.9, alternative = "greater") Difference of proportion power calculation for binomial distribution (arcsine transformation) h = 0.1518977 n = 742.3289 sig.level = 0.05 power = 0.9 alternative = greater NOTE: same sample sizes So those are very similar to those from Stata. h = 2arcsin(sqrt(p1)-2arcsin(sqrt(p2)), where p1=0.15 and p2=0.10 $\endgroup$
    – Marcin
    Nov 8, 2014 at 20:03
  • $\begingroup$ @MarcinKosinski I edited the answer to address your split question. $\endgroup$
    – dimitriy
    Nov 8, 2014 at 20:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.