Sample size determination for comparing two independent samples with regards to several continuous variables

Question

I am interested in calculating the required sample size to perform several statistical tests on the same sample.

More precisely, this is the situation:

There are two groups of individuals, A and B, and we want to test whether or not they show differences with regards to several continuous (hopefully normally distributed) variables.

So, basically, I want to perform several $t$ tests for mean comparison of independent samples.

Actually, focusing just on one of the variables (i.e., one of the multiple comparisons to perform), it is not clear to me how to calculate the sample size. As far as I know, usual methods to determine sample size in this case require to be provided with the following information:

• an initial guess of the (global) mean of the variable under study;

• an initial guess for the within-group standard deviation (assuming that it is the same in both groups!);

• a hypothesised value for the difference of the means between those two groups (basically, the null hypothesis) —normally zero;

• an estimation of the proportion of individuals in groups A and B;

• a desired confidence level / a significance level;

• a desired power to detect

• a given difference between the two means.

Is this right?

Finally, I am also concerned about exact sample size calculation, instead of an approximation based on asymptotic behaviours.

So, my question is two- (or maybe three-) fold:

1) What is an appropriate formula for calculating the required sample size in this context (for a one single comparison)?

2) Can be the previous formula extended to the multiple tests case? How should I exactly deal with multiple tests in this case? The concrete context, as explained above, is: performing several $t$ tests on the same pair of samples, to compare means for different variables. I guess there is a sort of correction to be done in order to ensure a real significance level but, anyway, should I take the maximum of all the individually calculated sample sizes? If not, what should be done?

3) How do I take into account exact tests in this context?

I know that this subject has been previously addressed in this website (see Determining sample size using power analysis in multiple testing context, for instance), but it is still not clear to me how to deal with the different sample sizes that I obtain for each $t$ test to be performed.

Answer 1

Your propositions are mostly correct, although you don't really need to have developed an estimate of the global means, and you do need a defined alpha level (probably .05, unless you plan to adjust for multiple testing).

a) It's probably easiest just to use a program like g*power for a simple independent groups t-test, (www.gpower.hhu.de/en.html - free and easy to install) or use pwr.t.test if you use R.

b) Power generally refers to the power of an analyses (e.g., a particular t test given the true population effect size). Calculate power for the smallest effect you are interested in, and ensure you gave enough participants/a large enough sample to be adequately powered to reliably detect an effect of that size. The tests performed for all of the other variables will have higher levels of power if their effect sizes are larger (and there's no missing data). If you adjust your alpha for multiple comparisons use the adjusted alpha level in your power calculation.