If I have a normally distributed population of 3.5 million elements, and I want to sample enough of them to make a statement with 99.6% confidence on a 1-tailed test, what should my sample size be? Is 100 enough, or would it need to be significantly larger?
You can use the following R script to generate your required sample:
pwr.t.test(n = SAMPLE_SIZE , d = EXPECTED_EFFECT_SIZE, sig.level = 0.004 , power =
DESIRED_POWER, type = c("one.sample"), alternative = "greater"/"less")
You need to provide 3 of the 4 values (n,*d*,sig.level, and power), while making the 4th value =
NULL. For example, if you wanted to see if 100 subjects is enough, you could fill in information for n, d (if you have an expected effect size, for this example d will be equal to 0.4), your significance level of 0.004 (or 99.6% CI), while leaving power as
Null. Lastly, you will need to choose the direction for the one-tailed test by choosing "greater" or "less", which for this example, I chose to use "greater". The below R script will perform this calculation:
pwr.t.test(n = 100 , d = 0.4, sig.level = 0.004 , power = NULL, type = c("one.sample"), alternative = greater)
Below is the generated output:
One-sample t test power calculation n = 100 d = 0.4 sig.level = 0.004 power = 0.8991141 alternative = greater
As you can see in the above output, we have 89.9% power to test our null hypothesis with an expected difference in effect size of 0.4 with 99.6% confidence ($a$ = 0.004).