1
$\begingroup$

Suppose we have the null and alternative hypotheses regarding a proportion

$H_0: p = p_0$

$H_a: p \ne p_0$

If we assume that $p$ takes the value $p_1$, what sample size do we need for a z-test of the null hypothesis at the $\alpha$ level with power $1 - \beta$?

$\endgroup$
5
  • 3
    $\begingroup$ What test do you plan to use? The power depends on that. $\endgroup$
    – whuber
    Commented Nov 22, 2021 at 22:00
  • 2
    $\begingroup$ What is $p_1$? What is $p_0$? The power depends on that. $\endgroup$ Commented Nov 22, 2021 at 22:55
  • $\begingroup$ @gung-ReinstateMonica arbitrary numbers between 0 and 1. $\endgroup$ Commented Nov 23, 2021 at 0:02
  • $\begingroup$ Then you will need arbitrarily different sample sizes. $\endgroup$ Commented Nov 23, 2021 at 1:28
  • $\begingroup$ @gung-ReinstateMonica obviously $\endgroup$ Commented Nov 23, 2021 at 1:32

2 Answers 2

1
$\begingroup$

The z-test based on the normal approximation rejects the null when the test statistic is greater than $z_{1 - \alpha/2}$,

$\big|{\frac{\hat{p} - p_0}{\sqrt{p_0 (1 - p_0) / n}}\big|} > z_{1-\alpha/2}$

Or, equivalently,

$\big|{\frac{\hat{p} - p_1}{\sqrt{p_1 (1 - p_1) / n}}\frac{\sqrt{p_1 (1 - p_1)}}{\sqrt{p_0 (1 - p_0)}} - \frac{p_1 - p_0}{\sqrt{p_0 (1 - p_0) / n}}\big|} > z_{1-\alpha/2}$

With $z = \frac{\hat{p} - p_1}{\sqrt{p_1 (1 - p_1) / n}}$, we have

$\big| z \frac{\sqrt{p_1 (1 - p_1)}}{\sqrt{p_0 (1 - p_0)}} - \frac{p_1 - p_0}{\sqrt{p_0 (1 - p_0) / n}}\big| > z_{1-\alpha/2}$

or equivalently,

$\big| z - \frac{p_1 - p_0}{\sqrt{p_1 (1 - p_1)/n}}\big| > z_{1-\alpha/2} \frac{\sqrt{p_0 (1 - p_0)}}{\sqrt{p_1 (1 - p_1)}}$

When $p = p_1$, $z \sim N(0,1)$ asymptotically, and the asymptotic power of the test is therefore equal to

$1 - \beta = \Phi\left( \frac{\sqrt{n}|p_1 - p_0| - z_{1-\alpha/2}\sqrt{p_0 (1 - p_0)}}{\sqrt{p_1 (1 - p_1)}} \right) + \Phi\left( \frac{-\sqrt{n}|p_1 - p_0| - z_{1-\alpha/2}\sqrt{p_0 (1 - p_0)}}{\sqrt{p_1 (1 - p_1)}} \right)$

Typically the second term on the right of the equation is quite small, and so we can approximate

$1 - \beta = \Phi\left( \frac{\sqrt{n}|p_1 - p_0| - z_{1-\alpha/2}\sqrt{p_0 (1 - p_0)}}{\sqrt{p_1 (1 - p_1)}} \right)$

Taking the inverse of the normal on both sides,

$z_{1-\beta} = \frac{\sqrt{n}|p_1 - p_0| - z_{1-\alpha/2}\sqrt{p_0 (1 - p_0)}}{\sqrt{p_1 (1 - p_1)}} $

Solving for $n$,

$n = \left(\frac{z_{1-\beta}\sqrt{p_1 (1 - p_1)} + z_{1-\alpha/2}\sqrt{p_0 (1 - p_0)}}{p_1 - p_0}\right)^2 $

$\endgroup$
1
$\begingroup$

One approximation to the power function is to use the p-value function. The p-value for a score test of $H_0: p\le p_0$ vs $H_a: p > p_0$ is

$$H(p_0;\hat{p})=1-\Phi\bigg(\frac{\hat{p}-p_0}{\sqrt{p_0(1-p_0)/n}}\bigg).$$

At the minimum detectable effect $\hat{p}_{MDE}$ the p-value testing $H_0:p\le p_0$ equals the alpha level of the test, $H(p_0;\hat{p}_{MDE})=\alpha$. For a given $n$ this p-value function evaluated at any other value $p$ is approximately equal to the power of the test,

$$1-\beta(p)\approx H(p;\hat{p}_{MDE})=1-\Phi\bigg(\frac{\hat{p}_{MDE}-p}{\sqrt{p(1-p)/n}}\bigg),$$

where the minimum detectable effect is $\hat{p}_{MDE}=p_0+z_{1-\alpha}\sqrt{p_0(1-p_0)/n}$.

This can be numerically solved for the sample size $n$. This approach can be simplified even further by inverting a Wald test,

$$1-\beta(p)\approx H(p;\hat{p}_{MDE})=1-\Phi\bigg(\frac{\hat{p}_{MDE}-p}{\sqrt{\hat{p}_{MDE}(1-\hat{p}_{MDE})/n}}\bigg),$$

where the minimum detectable effect is $\hat{p}_{MDE}=p_0+z_{1-\alpha}\sqrt{\hat{p}_{MDE}(1-\hat{p}_{MDE})/n}$. This is particularly useful in more complicated model. Of course for a binomial proportion the power of a test can be solved exactly by identifying the rejection region of the test and then evaluating the complementary CDF of the binomial distribution under the alternative.

Here is a paper discussing inference on power as well as approximating power using a p-value function.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.