power and sample size estimation

Question

For a one-sided test: $$H_0: \mu_1 \leq \mu_2 \ H_1: \mu_1 >\mu2$$

$$\alpha=0.025,\beta=0.1$$

How to calculate the sample size needed? Is this information enough to calculate the sample size?

Answer 1

Suppose that this is a one-sided, one-sample t test. You want to detect whether $\mu_2 - \mu_1 = \Delta = 2$ or more and you estimate that the population standard deviation is $\sigma = 10.$ You are working at level 2.5% and want power 90% $(\beta = 0.1).$

Minitab, R, and many other statistical programs (and some on-line sites) have 'power and sample size' procedures that allow you to find the sample size implied by the constraints mentioned above. (These program implement a formula that uses the non-central t distribution.)

It turns out that the quantity $\Delta/\sigma$ is used in the formula, so if all else remains the same, $\Delta =1, \sigma = 5$ would lead to the same sample size: $n=199$ in the current problem.

Here is a relevant printout from Minitab:

Power and Sample Size 

1-Sample t Test

Testing mean = null (versus > null)
Calculating power for mean = null + difference
α = 0.025  Assumed standard deviation = 10

            Sample  Target
Difference    Size   Power  Actual Power
         2     199     0.8      0.801690

We can use simulation of $100\,000$ experiments in R to verify that $n = 199, \Delta = 2, \sigma = 10, \alpha=0.025,$ lead to a simulated power of about 80%.

set.seed(2022)
pv = replicate(10^5, 
      t.test(rnorm(199,100,10), mu=98, alt="g")$p.val)
mean(pv <= 0.025)
[1] 0.8031  # aprx power 80%

Changing $\Delta$ and $\sigma$ as mentioned above.

set.seed(522)
pv = replicate(10^5, 
      t.test(rnorm(199,100,5), mu=99, alt="g")$p.val)
mean(pv <= 0.025)
[1] 0.80179