8
$\begingroup$

I would like to do power analyses for hypothesis tests of (non-)equality of proportions in which the proportions are very small. I would like to do so without using normal (or Poisson) approximations of the binomial distribution. There are several general types of power questions I'd like to be able to address.

  1. Post-hoc: Given $\Pr_1$ (probability of a success in group 1) and $\Pr_2$ and $N_1$ (sample size group 1) and $N_2$ to calculate the power of the design given $\alpha$.
  2. A priori solve for $N$ given $\alpha$, the ratio $N_1\over{N_2}$, $1 - \beta$ (power), $\alpha$, $\Pr_1$, and an expected $\Pr_2$
  3. A priori solve for $1 - \beta$ given $\alpha, N_1, N_2, \Pr_1$, and $\Pr_2$.

An ideal response would include R code and point out any other givens that I forgot to point out. A simulation approach is not a suitable response due to the small proportions. With your solution, please also mention what kind of statistical test it is applicable to.

Improve this question
$\endgroup$
7
  • $\begingroup$ One very general way to deal w/ these is via simulation. See, e.g., my answer here: simulation of logistic regression power analysis designed experiments. $\endgroup$
    – gung - Reinstate Monica
    Commented May 26, 2013 at 16:42
  • 1
    $\begingroup$ I didn't think simulations would be a fruitful approach when proportions are very small (on the order of 1 in 50,000 or less). $\endgroup$
    – russellpierce
    Commented May 26, 2013 at 17:34
  • $\begingroup$ I'm also not clear on how hypothetical instances where a marginal proportion is 0 should be treated. $\endgroup$
    – russellpierce
    Commented May 26, 2013 at 17:39
  • $\begingroup$ @RusselS.Pierce, you're right, simulations become more unwieldy the closer the proportions get to the boundaries. That's because power (viz, whether an iteration is 'significant') is a Bernoulli. $\endgroup$
    – gung - Reinstate Monica
    Commented May 26, 2013 at 17:44
  • $\begingroup$ This is relevant: stats.stackexchange.com/questions/235750/… how small do you imagine $p$ can be? $\endgroup$
    – kjetil b halvorsen ♦
    Commented Apr 14, 2017 at 12:31

1 Answer 1

Reset to default
1
$\begingroup$

This is not an answer. It is a community wiki that people may edit as they look for the answer.

G*power 3 can perform (approximations) of these analyses (per this site). The canonical reference for that software provides a reference for performing (at least some) of these types of power analyses as Cohen, 1988 chapter 6 (and 7) as does this example using SAS. The exact equations/procedures may be available from that source. However, the approximations appear to break down at small probabilities.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Why isnt' this an answer? It looks like one to me... $\endgroup$
    – gung - Reinstate Monica
    Commented May 26, 2013 at 16:39
  • 1
    $\begingroup$ I suppose it is, the GPower reference does claim that "Cohen’s(1988) effectsize g is used and exact power values based on the binomial distribution are calculated". I suppose I was just holding out hope for actual equations. $\endgroup$
    – russellpierce
    Commented May 26, 2013 at 17:35
  • $\begingroup$ It actually isn't. The referred to claim is not relevant to the 2 groups binomial case. $\endgroup$
    – russellpierce
    Commented Jun 10, 2013 at 23:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.