5
$\begingroup$

I am trying to do a priori sample size calculation based on published results. However, I am unable to obtain a reasonable estimate of the published effect size (which is not reported) as I am unable to obtain an estimate of the pooled standard deviation or the standard deviation of the difference.

The problem resides in the fact that it is a factorial experiment with two within-subjects factors ($2 \times 3$ levels). I only have the cell means and standard deviations (i.e., for the $2 \times 3$ table) but not the marginal means for the first factor (with 2 levels) which I need.

I know that the formula for the pooled standard deviation for independent samples (taken from wikipedia) is:

$$s_p = \sqrt{\frac{(n_1-1) s_1^2 + (n_2-1) s_2^2 + \cdots + (n_k-1) s_k^2}{n_1 + n_2 + \cdots + n_k - k}}.$$

But what is the formula for pooled standard deviation for dependent samples?


Means:

     1A   1B
2a 3.24 3.01
2b 2.91 2.56
2c 3.01 3.05

Standard deviations:

     1A   1B
2a 0.65 0.70
2b 0.68 0.60
2c 0.46 0.53

I want to obtain the effect size between 1A and 1B (so pool over levels of factor 2). Sample size is 27.

$\endgroup$

1 Answer 1

6
$\begingroup$

If this is a completely balanced within-subject design with $27\times 6=162$ observations, then you can actually calculate the marginal means: simply average over the levels of the second factor. Of course, you have to be sure that averaging over different conditions is meaningful for your planned experiment - do you expect each of those conditions to be present with about 1/3 probability?

The real difficulty is with the variance of the difference. It is well known that $$Var(X-Y) = Var(X) + Var(Y) - 2 SD(X)SD(Y)Corr(X,Y)$$ The problem is that you don't know the within-subject correlation.

Option 1. You could just to guess at a value: would you expect the correlation to be high or low? Since higher correlation will lead to lower variance, you could assume the worst case scenario of 0 correlation, and be guaranteed to overestimate the required sample size (unless the true correlation is negative, but that is rare).

Option 2. If the published results have more information, like a p-value from a test, you could try to figure out the correlation. For a complicated design like this one, it might be difficult to do analytically, but you could try a simulation approach. Given a correlation coefficient, simulate data with the given means and variances, run the test and check the p-value. Modify the correlation coefficient until you get close to the published result.

$\endgroup$
3
  • $\begingroup$ Thanks a lot. I actually have some estimate of the correlation (we replicated the original study and used our correlation). From this I ended up using the mean of the sd (i.e., totally ignoring the $n$ in the formula) as it was indeed completely balanced. With means, sds and correlation you can make a power analysis using the fabulous G*Power 3: psycho.uni-duesseldorf.de/abteilungen/aap/gpower3 $\endgroup$
    – Henrik
    Apr 16, 2012 at 14:58
  • $\begingroup$ Are you sure it is Corr and not Cov? $\endgroup$ Jan 18, 2017 at 11:24
  • 1
    $\begingroup$ $Cov(X,Y) = SD(X) SD(Y) Corr(X,Y)$, so you can write the formula with either covariance or correlation. $\endgroup$
    – Aniko
    Jan 23, 2017 at 15:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.