Power calculation for two populations of equal variance

Question

If I had a scientific assay which when run on a control population produces readouts which are normally distributed about the mean. A treatment which changes this mean does not change the variance of the population, just the mean. That is to say any treatment simply changes the population mean but not the shape of the bell curve.

I can calculate the SD ect for the control population and would like to predict the smallest change in mean which would be significant at p=0.05 for a given population size. What power calculation can be used to predict this?

E.g. If I had 100 samples in each group, control mean of 10 with an SD of 1 what value of the second "experimental" population would give p<0.05 when the two populations are compared with a t-test?

Answer 1

Comment continued: Here is an example of the simulation method in R. For $n = 100, \Delta = 4, \sigma = 10,$ the power of a two-sided t test at level $\alpha = 0.05$ is above 97% $(0.977 \pm 0.001).$

set.seed(2021)
n = 100;  Dlt = 4;  sgm = 10
p.val = replicate(10^5, t.test(rnorm(n,Dlt,sgm),mu=0)$p.val)
mean(p.val <= .05)
[1] 0.97702              # aprx power
2*sd(p.val <= .05)/sqrt(10^5)
[1] 0.0009476739         # aprx 95% margin of sim error

For your example with $n=100, \sigma=1,$ with a two-sided t test at level $\alpha=0.05,$ you would have a very good chance of detecting a change in population mean of about $\Delta=0.5.$

set.seed(1207)
n = 100;  Dlt = .5;  sgm = 1
p.val = replicate(10^5, t.test(rnorm(n,Dlt,sgm),mu=0)$p.val)
mean(p.val <= .05)
[1] 0.99863

Here is one such test:

set.seed(1234)
x = rnorm(100, .5, 1)
t.test(x, mu=0)

        One Sample t-test

data:  x
t = 3.4173, df = 99, p-value = 0.000919
alternative hypothesis: 
  true mean is not equal to 0
95 percent confidence interval:
 0.1439425 0.5425341
sample estimates:
 mean of x 
 0.3432383 

Note: You might want to look at some of the 'Related' pages on this site, linked in the margin of this page.