1
$\begingroup$

Say a survey is taken where the options are yes and no. We can take the values to correspond to $1$ and $0,$ if this is done for each person, $n$ (our sample size) Bernoulli distributions are created but if put together form a binomial distribution. So now I take $Y_n$ as the sum of all $1$'s with each $1$ having probability $p$ of occurring. I wish to turn this into a hypothesis test of form

$H_0: p=p_0$ against $H_1: p \ne p_0,$ with significance level = $\alpha.$

I wish to find the minimal sample size $n$ such that the hypothesis test yields an accurate result. Can anyone help?

Tldr: I need to find a way of minimising the sampling size for a binomial test for proportions such that the hypothesis test holds for a power of beta (0.8) as an example

$\endgroup$
1
  • $\begingroup$ Some additional information is needed to find the power of the power of your test. See my Answer. // Quibble: Check your notation, symbol $\beta$ 'beta' is usually used for the probability of 'Type II Error' and, for a particular alternative, power is $1$ minus the probability of Type II error. $\endgroup$
    – BruceET
    Commented Apr 20, 2021 at 21:23

1 Answer 1

2
$\begingroup$

In order to answer this question, you need to specify how large a difference from $p_0$ you want to be 80% sure to detect.

Below is a printout from the 'power and sample size' procedure for one-sample tests of proportion in Minitab statistical software. I assumed a test of $H_0: p = .5$ against $H_a: p \ne .5,$ at the 0.05 = 5% level of significance with power .8 = 80% for specific alternative values $p_a = .6, .7, .8.$

    Power and Sample Size 

Test for One Proportion

Testing p = 0.5 (versus ≠ 0.5)
α = 0.05

              Sample  Target
Comparison p    Size   Power  Actual Power
         0.6     194     0.8      0.800314
         0.7      47     0.8      0.803325
         0.8      20     0.8      0.817041

As you can see, if your null success value is $p_0 = .5,$ then it is easier to detect that the success probability is truly $p_a = 0.7$ than to detect $p_a = .6.$ So you need a smaller sample size for the former than for the latter. The following power curve from Minitab, shows powers for various differences between $p_0 = .5$ and $p_a$ for each choice of sample size. Other statistical software programs (and with varying degrees of clarity and accuracy, also some pages online) give similar output.

enter image description here

Notes: (1) In these computations, it is not just the distance from the null value $p_0$ that matters, but also the null value itself. For example, if $p_0 = .2,$ then the sample sizes necessary to detect specific alternative values $p_a = .3, .4, .5$ are $n = 137, 36, 17,$ respectively. [Minitab output not shown.]

(2) With a sample size as large as $n = 194,$ a normal approximation to the binomial distribution is good. So it would not matter whether you use an exact binomial test or an approximate normal version of the test. [You could find the critical values using standardization, solving quadratic equations in $n$ for closest integer values, and printed tables of the standard normal CDF. And then, more easily, you could verify the power.]

(3) Specifically, for testing $H_0: p = .5$ against $H_a: p > .5$ the critical values for a test at level $\alpha = 0.5$ are $83$ and $111.$ That is $$P(X \le 83|p=.5)+P(X\ge 111|p=.5) \approx 0.05 = 5\%.$$ Also, such a test gives power $0.8 = 80\%$ against the alternative value $p_a = 0.6$ because $$P(X \le 83|p=.6)+P(X\ge 111|p=.6) \approx 0.8 = 80\%.$$ [Because the binomial distribution is discrete, it is not possible for a nonrandomized test to have significance level exactly 5% or a power against a specific alternative of exactly 80%.] Computations in R where dbinom is a binomial PDF:

sum(dbinom(c(0:83, 111:194), 194, .5))
[1] 0.05228312
sum(dbinom(c(0:83, 111:194), 194, .6))
[1] 0.8067075

In the figure below, the significance level of the test is the sum of the heights of the blue bars in the rejection region outside the vertical dotted lines. The power of the test against the specific alternative $p_a = 0.6$ is the sum of the heights of the brown bars in the rejection region---mainly in the the right tail. [Theoretically, both binomial distributions take values from 0 through 194, however probabilities of values of both distributions below 60 and above 140 are negligible and are not shown.]

enter image description here

R code for figure:

k =60:140; PDF = dbinom(k, 194, .5)
pdf.a = dbinom(k, 194, .6)
hdr = "PDFs of BINOM(194, .5) [blue] abd BINOM(194, .6)"
plot(k-.1, PDF, xlab="x", type="h", lwd=2, col="blue", main=hdr)
 lines(k+.1, pdf.a, type="h", lwd=2, col="brown", main=hdr)
  abline(h=0, col="green2")
  abline(v=c(83.5,110.4), lty="dotted")
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.