8
$\begingroup$

I would like to do power analyses for hypothesis tests of (non-)equality of proportions in which the proportions are very small. I would like to do so without using normal (or Poisson) approximations of the binomial distribution. There are several general types of power questions I'd like to be able to address.

  1. Post-hoc: Given $\Pr_1$ (probability of a success in group 1) and $\Pr_2$ and $N_1$ (sample size group 1) and $N_2$ to calculate the power of the design given $\alpha$.
  2. A priori solve for $N$ given $\alpha$, the ratio $N_1\over{N_2}$, $1 - \beta$ (power), $\alpha$, $\Pr_1$, and an expected $\Pr_2$
  3. A priori solve for $1 - \beta$ given $\alpha, N_1, N_2, \Pr_1$, and $\Pr_2$.

An ideal response would include R code and point out any other givens that I forgot to point out. A simulation approach is not a suitable response due to the small proportions. With your solution, please also mention what kind of statistical test it is applicable to.

$\endgroup$
  • $\begingroup$ One very general way to deal w/ these is via simulation. See, e.g., my answer here: simulation of logistic regression power analysis designed experiments. $\endgroup$ – gung May 26 '13 at 16:42
  • 1
    $\begingroup$ I didn't think simulations would be a fruitful approach when proportions are very small (on the order of 1 in 50,000 or less). $\endgroup$ – russellpierce May 26 '13 at 17:34
  • $\begingroup$ I'm also not clear on how hypothetical instances where a marginal proportion is 0 should be treated. $\endgroup$ – russellpierce May 26 '13 at 17:39
  • $\begingroup$ @RusselS.Pierce, you're right, simulations become more unwieldy the closer the proportions get to the boundaries. That's because power (viz, whether an iteration is 'significant') is a Bernoulli. $\endgroup$ – gung May 26 '13 at 17:44
  • $\begingroup$ This is relevant: stats.stackexchange.com/questions/235750/… how small do you imagine $p$ can be? $\endgroup$ – kjetil b halvorsen Apr 14 '17 at 12:31
1
$\begingroup$

This is not an answer. It is a community wiki that people may edit as they look for the answer.

G*power 3 can perform (approximations) of these analyses (per this site). The canonical reference for that software provides a reference for performing (at least some) of these types of power analyses as Cohen, 1988 chapter 6 (and 7) as does this example using SAS. The exact equations/procedures may be available from that source. However, the approximations appear to break down at small probabilities.

$\endgroup$
  • $\begingroup$ Why isnt' this an answer? It looks like one to me... $\endgroup$ – gung May 26 '13 at 16:39
  • 1
    $\begingroup$ I suppose it is, the GPower reference does claim that "Cohen’s(1988) effectsize g is used and exact power values based on the binomial distribution are calculated". I suppose I was just holding out hope for actual equations. $\endgroup$ – russellpierce May 26 '13 at 17:35
  • $\begingroup$ It actually isn't. The referred to claim is not relevant to the 2 groups binomial case. $\endgroup$ – russellpierce Jun 10 '13 at 23:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.