4
$\begingroup$

I am comparing two power functions in R, a base function stats::power.prop.test, and the function pwr::power.2p.test in the pwr package.

I would think that they would give the same answer, but they are slightly different. And the equations being solved in the source code are different. Why are they different, and in what circumstances should I use one over the other?

If I understand the source code right, here is the power formula when the alternative hypothesis is that $p_1 < p_2$ for pwr:

$$ \Phi \left( \Phi^{-1}\left( \alpha \right) - 2 \left( \arcsin \sqrt{p_1} - \arcsin \sqrt{p_2} \right) \sqrt{\frac{n}{2}}\right) $$

and for stats:

$$ \Phi \left( \frac{ \sqrt{n} \left| p_1 - p_2 \right| + \Phi^{-1} \left( \alpha \right) \sqrt{ \left( p_1 + p_2 \right) \left( 1 - \left( p_1 + p_2 \right) \right)} } { \sqrt{p_1 \left( 1 - p_1 \right) + p_2 \left( 1 - p_2 \right)} }\right) $$

Below is some code comparing the output for the two functions.

require(pwr)
stats::power.prop.test(p1 = .50, p2 = .75, power = .90, 
  sig.level = 0.05, alternative = "two.sided")
## gives n = 76.71
pwr::pwr.2p.test(h = ES.h(0.50,0.75), power = .90, 
  sig.level = 0.05, alternative = "two.sided")
## gives n = 76.65
$\endgroup$

1 Answer 1

5
$\begingroup$

Both calculations are based on normal approximations; the second on the normal approximation to the binomial, the first on the variance-stabilizing transformation for a binomial proportion.

Both are approximate, so you wouldn't necessarily expect them to give the same answer as each other, or as the actual binomial calculation.

It's likely that the first will tend to work well over a somewhat wider range of $p$ at a given $n$.

If you really care about accuracy, you can use either one as a "first guess" and see (via simulation if you can't do the calculations) if you get the desired power using the exact binomial test rather than an approximation, and adjust sample size (in most cases, probably only by 1 or 2) if it isn't quite there.

(NB I haven't checked the actual formulas that you have are correctly done, I am simply discussing them based on recognizing their general form.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.