I am designing an experiment and want to define the sample size. To identify this, I am setting my significance level to 0.05 and the power to 0.8. My alternative hypotheses says the two means should be at least 15% different from each other. Does this 15% correspond to the effect size? I had a look at this and this, but I am still not sure how to interpret this 15% I am setting my alternative hypothesis in the context of power analysis.
I tried to calculate the sample size based on the assumption that the 15 % is the effect size:
from statsmodels.stats.power import TTestIndPower # parameters for power analysis effect = 0.15 alpha = 0.05 power = 0.9 # perform power analysis analysis = TTestIndPower() result = analysis.solve_power(effect, power=power, nobs1=None, ratio=1.0, alpha=alpha) print('Sample Size: %.3f' % result)
The output is
Sample Size: 934.954
This does not seem reasonable. I am not sure if I a doing it the right way.
Can someone help here?