If I had a scientific assay which when run on a control population produces readouts which are normally distributed about the mean. A treatment which changes this mean does not change the variance of the population, just the mean. That is to say any treatment simply changes the population mean but not the shape of the bell curve.

I can calculate the SD ect for the control population and would like to predict the smallest change in mean which would be significant at p=0.05 for a given population size. What power calculation can be used to predict this?

E.g. If I had 100 samples in each group, control mean of 10 with an SD of 1 what value of the second "experimental" population would give p<0.05 when the two populations are compared with a t-test?

  • $\begingroup$ (1) A math stat book or advanced applied one should have a formula to find the power of a t test at level $\alpha=0.05$ in terms of $n$ and $\Delta/\sigma,$ where $\sigma$ is the population SD and $\Delta$ is the change in $\mu$ to be detected. This formula uses the non-central t distribution. (2) Some online 'power and sample size' procedures give useful results. (3) An R library has power and sample size procedures for this an other commonly used tests. (4) You can get a good approx, by doing a simulation with something like $10^5$ tests. $\endgroup$
    – BruceET
    Dec 8, 2021 at 2:38

1 Answer 1


Comment continued: Here is an example of the simulation method in R. For $n = 100, \Delta = 4, \sigma = 10,$ the power of a two-sided t test at level $\alpha = 0.05$ is above 97% $(0.977 \pm 0.001).$

n = 100;  Dlt = 4;  sgm = 10
p.val = replicate(10^5, t.test(rnorm(n,Dlt,sgm),mu=0)$p.val)
mean(p.val <= .05)
[1] 0.97702              # aprx power
2*sd(p.val <= .05)/sqrt(10^5)
[1] 0.0009476739         # aprx 95% margin of sim error

For your example with $n=100, \sigma=1,$ with a two-sided t test at level $\alpha=0.05,$ you would have a very good chance of detecting a change in population mean of about $\Delta=0.5.$

n = 100;  Dlt = .5;  sgm = 1
p.val = replicate(10^5, t.test(rnorm(n,Dlt,sgm),mu=0)$p.val)
mean(p.val <= .05)
[1] 0.99863

Here is one such test:

x = rnorm(100, .5, 1)
t.test(x, mu=0)

        One Sample t-test

data:  x
t = 3.4173, df = 99, p-value = 0.000919
alternative hypothesis: 
  true mean is not equal to 0
95 percent confidence interval:
 0.1439425 0.5425341
sample estimates:
 mean of x 

Note: You might want to look at some of the 'Related' pages on this site, linked in the margin of this page.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.