# power analysis for binomial test

I am trying to test whether the sex ratio of some sampled individuals significantly differs from the expected sex ratio of 1. I have n= 64, of which female=34 and male=30.

I ran a binomial test:

succ <- c(34,30)
binom.test(succ,0.5)

data:  succ
number of successes = 34, number of trials = 64, p-value = 0.708
alternative hypothesis: true probability of success is not equal to 0.5
95 percent confidence interval:
0.4023098 0.6572035
sample estimates:
probability of success
0.53125


I would like to calculate the statistical power of this test, and I know that power = 1-β, where β is the type II error.

I am getting confused when reading this explanation. I don't understand how to adapt this formula (for different choices of n) to my case:

enn = 1:2000
critical = qbinom(.025, enn, .5)
beta = pbinom(enn-critical,enn,.55) - pbinom(critical-1,enn,.55)


What I did was

1-(pbinom(34,64,0.5)- pbinom(30, 64, .5))
[1] 0.7410237


but I am not sure if it is correct to use 0.5 as probability. Moreover, I tried a different method, and I get a completely different result

pwr.p.test(ES.h(.53125,.5),n=64, power=NULL, alternative = "two.sided")

proportion power calculation for binomial distribution (arcsine transformation)

h = 0.06254076
n = 64
sig.level = 0.05
power = 0.07913605
alternative = two.sided


Is one of these two tests correct and why?

In order to find 'power', you need to have a specific alternative in mind. Suppose your null hypothesis is $$H_0: p = 0.5$$ vs. $$H_a: p > 0.5,$$ where $$p = P(\mathrm{Female}).$$ Also suppose you have $$n = 64$$ and you want the power of a test at level $$\alpha = 0.05$$ against the specific alternative $$p = 0.6.$$

For an exact binomial test, you need to find the critical value $$c$$ such that $$P(X \ge c\,|\,n=64, p=.5)$$ is maximized, but still below $$0.05.$$ In R, where dbinom, pbinom, and qbinom denote binomial PDF, CDF, and quantile function (inverse CDF), respectively, we see that the critical value is $$c = 40.$$ Notice that, because of the discreteness of binomial distributions, the so-called 5% level' actually rejects with probability $$P(\mathrm{Rej}\, H_0 | H_0\, \mathrm{True}) \approx 3\%.$$

qbinom(.95, 64, .5)
[1] 39
sum(dbinom(39:64, 64, .5))
[1] 0.05171094
sum(dbinom(40:64, 64, .5))
[1] 0.02997059
1 - pbinom(39, 64, .5)
[1] 0.02997059


Then the power of this test against alternative value $$p = 0.6$$ is given by $$P(X \ge 40\,|\,n=64, p=0.6) = 0.3927.$$

1 - pbinom(39, 64, .6)
[1] 0.392654


We can make a 'power curve' for this test by looking at a sequence of alternative values p.a between $$0.5$$ and $$.75.$$ The first block of R code below makes the solid black line in the plot below.

p.a = seq(.50, .75, by=.01)
p.rej = 1 - pbinom(39, 64, p.a)
plot(p.a, p.rej, type="l", main="Power Curve")
abline(h=c(.03,1), col="green2")


If we look at a level $$\alpha = 0.05$$ test of $$H_0: p = 0.5$$ vs $$H_a: p > 0.5$$ with $$n = 256$$ subjects, then the critical value is $$c = 141,$$ the rejection probability when $$H_0$$ is true is $$0.046,$$ and the power against various alternative values of $$p$$ is greater, as shown by the dotted blue line in the figure.

c.256 = qbinom(.95, 256, .5); c.256
[1] 141
1 - pbinom(c.256, 256, .5)
[1] 0.04565604
p.rej.256 = 1 - pbinom(c.256, 256, p.a)
lines(p.a, p.rej.256, col="blue", lty="dotted")


Notes: Because $$n = 64$$ is sufficiently large to use normal approximations, you might want to try using normal approximations. A disadvantage is that this ignores the issue of discreteness, so it may appear that your test rejects exactly 5% of the time when $$H_0$$ is true. Also, you'd need to use a continuity correction for best results.

One relevant computation for the significance level in R is:

1 - pnorm(39.5, 32, 4)
[1] 0.03039636


(Approximate) power is $$0.3895:$$

mu.a = 64*.6;  sg.a = sqrt(64*.6*.4)
mu.a; sg.a
[1] 38.4
[1] 3.919184

1 - pnorm(39.5, mu.a, sg.a)     # Using NORM(mu.a, sg.a)
[1] 0.3894815
1 - pnorm((39.5 - mu.a)/sg.a)   # Standardizing and using NORM(0,1).
[1] 0.3894815
`

It's also important to ask yourself why you are calculating the power. Since you already have the data, you are calculating a 'post-hoc' power statistic (as opposed to a pre-hoc power statistic). It's worth nothing that a number of authors have criticised the use of post-hoc power statistics from a Frequentist point of view - see, for example, https://gpsych.bmj.com/content/32/4/e100069