2
$\begingroup$

I ran a power simulation for a linear mixed model with an interaction of a continuous predictor and a predictor of condition (3 conditions), their intercepts and a varying intercept for subjects. Now I thought about having 1 or 3 trials per condition and ran simulations for both experiments. The estimated power did not really differ but I read that both, trial amount and sample size, affect power.

I now wonder how to balance the number of participants and the number of trials to get a well powered study. Any helpful hints?

$\endgroup$
3
  • $\begingroup$ Higher power comes from lower intra-cluster correlation (higher within-cluster variance) within subject. Add more trials per subject will increase power but by not very much if the trials are almost redundant. $\endgroup$ Feb 2, 2022 at 13:07
  • $\begingroup$ @FrankHarrell Thank you very much for this comment. Could you explain what the mathematical relationship between the ICC and power is? (Or provide a link where this is explained.) $\endgroup$
    – Pearson
    Feb 7, 2022 at 18:17
  • 1
    $\begingroup$ My favorite paper on this is one that computes the effective sample size as a function of the correlation pattern: tandfonline.com/doi/abs/10.1198/tast.2009.08196 - big picture is that higher correlations = higher redundancy = lower effective sample size = lower power. $\endgroup$ Feb 7, 2022 at 22:52

1 Answer 1

2
$\begingroup$

It is better to have more participants than to have more trials per participant*.

(from the point of view of the accuracy of the measurements, more participants might be more costly or difficult sampling)

See for instance the simplified example estimating the mean of a population where the participants follow a normal distribution and measurements within participants follow a normal distribution as well.

The mean of a participant $i$ be distributed as

$$\mu_i \sim N(\mu,\sigma_{p})$$

and the measurement of a particular participant

$$X_{ij} \sim N(\mu_i, \sigma_{t})$$

Let's consider that we measure $n_p$ participants and for each participant $n_t$ trials.

Then the mean of observations of participant $i$ is distributed as

$$\bar{X}_{i} = \frac{1}{n_t}\sum_{j=1}^{n_t} X_{ij} \sim N\left(\mu_i, \sqrt{\sigma_{t}^2/n_t}\right)$$

The mean of the mean of the participants is distributed as:

$$\bar{X} = \frac{1}{n_p}\sum_{i=1}^{n_p} \bar{X}_{i} \sim N\left( 0, \sqrt{\sigma_{p}^2/n_p + \sigma_{t}^2/(n_p \cdot n_t)}\right)$$

  • Increasing the number of participants decreases both the variation $\sigma_p^2$ due to the distribution among the participants and the variation $\sigma_t^2$ due to the distribution within the participants.

  • Increasing the number of trials reduces only the variation $\sigma_t^2$ due to the distribution within the participants

If the within-participant variation $\sigma_t$ is dominant, then the difference between the effects of increasing number of trials and increasing number of participants will be closer to each other.


Intuitively you can see it as sampling $n_p$ 'error' terms for each participant and $n_p \cdot n_t$ 'error' terms for each trial in each participant. Your mean is an average over these terms, and the variance will be less the more you sample, but the number of trials does not reduce the variance of the mean of the $n_p$ terms for the between participant variation.


However if there are additional costs to obtaining a participant, and obtaining more trials is cheaper, then there might be some optimum where we need more trials per participant. See: Optimize the number of individuals inside a sample

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.