4
$\begingroup$

I am using g*power to calculate the a priori power for a 2 group repeated measures anova with a continuous dependent variable with 3 time points. I think that higher values for the "correlation among rep measures" should indicate a better test (i.e., high test-retest correltaion) and thus yield better power and a lower total sample size; however the opposite seems to be true:

Test family: F tests; Statistical test: ANOVA: repeated measures, between factors

Effect size f: 0.25 alpha error probability: 0.05 Power: 0.8 Number of groups: 2 Number of measurements: 3

Correlation among rep measures: 0.8 (a reasonable value for these neuropsychological assessments of cognitive function) -> total sample size: 112

Correlation among rep measures: 0.5 (a poor test would have a value this low) -> 86

What am I not understanding? Thanks!

Henry

$\endgroup$
2
$\begingroup$

Higher correlation within subject gets you more power when the test being done is a differencing, equivalent to a paired t-test. The standard deviation used in calculating effect size is multiplied by $\sqrt{1-\rho}$. The standard deviation for difference scores (for a one-sample test) is $SD\sqrt{2-2\rho}$. This also applies where it is appropriate to model using random intercepts, because that means a chunk of variability is being accounted for by individual random effects, and not being counted as error.

The opposite happens when the test is strictly between subjects. There you are in the world of increasing your effective sample size (as indicated by "Onestop" earlier). By repeating measurements, you increase your sample by the degree to which the observations are uncorrelated. The standard deviation for an average of independent (uncorrelated) numbers drawn from the same distribution is the standard error: $SD\sqrt{\frac{1}{N}}$. For averages of correlated numbers, it is roughly $SD\sqrt{\frac{1}{N}+\rho\frac{N-1}{N}}$. Note that as correlation approaches $1$, you get back the original $SD$, and as it approaches $0$, $SD$ approaches the $SE$ for independent observations.

Paul

$\endgroup$
  • $\begingroup$ OK - I think this is the right conceptual answer - I hadn't appreciated the difference between the effect of the correlation across repeated measures on the within group power (improved) and between groups power (degraded). Of course, given that this is a clinical trial, what I really care about is the power to detect a group x time interaction. I believe I should be using in g*power the "repeated measures, within-between interaction" model, which does show increasing power with increasing correlation over the repeated measure. $\endgroup$ – Henry Mahncke Sep 17 '10 at 23:11
2
$\begingroup$
  • All else being equal, bigger correlations in a repeated measures design means more power
  • A larger correlation means that more of the variance in the dependent variable is explained by systematic effects of cases/individuals
  • Prior research or previous datasets are useful for estimating the correlation. Estimates of test-retest reliability of measures is a starting point. Interestingly, I've found that greater changes in the means (i.e., the hypothesis of interest) is associated with slightly lower correlations than would otherwise be the case. Thus, as a very rough rule of thumb, you might want to reduce your estimate of the correlation a little bit from the published test-retest reliability estimates, perhaps by .1 or .2. Also, you might want to tweak the test-retest reliability estimate if the time between measurements is different to the published test-retest reliability estimates (i.e., increasing it if the time frame is shorter, and increasing it if it is longer).
  • Yes, more reliable tests will generally lead to larger test-retest reliability and thus, stronger correlations between levels of the repeated measures factor and more power. They should also lead to larger observed effect sizes (i.e., cohen's ds) because there is less attenuation for low reliability. Of course, theoretically the more you are interested in traits rather than states, the harder it will be to actually create meaningful changes.
$\endgroup$
1
$\begingroup$

Higher correlation among repeated measures means that each repeat is adding less information to the first measurement. If the correlation were 1, the repeats are no extra information to the original measurement, and the sample size could be calculated ignoring the repeats. If the correlation were 0, each repeated measurement of a subject adds the same amount of information as an extra subject, so the sample size could be calculated treating every observation as independent.

$\endgroup$
  • 1
    $\begingroup$ The mean at different levels of the repeated measures factor could change, even when r = 1.0 between scores at different time points. $\endgroup$ – Jeromy Anglim Sep 16 '10 at 3:14
1
$\begingroup$

I would guess the explanation is something like this: The higher the correlation between the time points, the lower the proportion of unshared variance. That is, the lower the proportion of variance that can contain the effect (i.e., the unshared variance). Given that in this unshared variance the ratio of effect to non-effect is the same (i.e., your effect size) the total amount of variance that contains the effect is smaller the higher the correlation between the time points. Hence, the higher the correlation the more power (i.e., sample size) needed to find the expected effect.

However, I think there is another issue in your question you have to think about. What does the effect size stand for? I am not a user of g-power, but i guess it is the effect size for the interaction of group by time points. In this case my explanation should apply but you should ask yourself if this is the effect you are interrested in.

Furthermore, you shouldn't confuse test-retest reliability with correlation between time-points. Generally you want a high test-retest reliability, but in most cases you want a small correlation between time points but an effect between the time point (i.e., rather low correlation).

Hope this is all clear, currently in a hurry, but can get extend/clarify tomorrow.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.