2
$\begingroup$

I have a repeated measures experiment with 5 factors each with 3 different levels. I'm trying to figure out a way to reduce the demand on participants by making them not have to sit through every condition.

I had been looking into part factorial designs, but it occurred to me that I could possibly run a multi-level model with partial completion for each participant.

So, my question is this: Is it possible to show each participant a random subset of conditions (say 50%), gather enough participants so there's good representation for each individual condition, and then use a multilevel linear model to establish the effects of each factor? I suspect it might be, but I'm not sure if there's anything I'm overlooking.

$\endgroup$

1 Answer 1

6
$\begingroup$

Yes, that should work fine provided you have enough participants.

You could convince yourself (without math) by simulating some data from such an experimental design and showing that you can recover the correct effects (you do need to know a bit about how these models are parameterized). In particular, if you're using R and have loaded the lme4 package, you can use

simulate( ~ f1 + f2 + f3 + f4 + f5 + (f1 + f2 + f3 + f4 + f5 | subject),
          newdata = dd,   ## experimental design
          newparams = list(beta = <vector of 1 + 5*2 = 11 parameters>,
                           theta = <vector of 11*12/2 = 66 parameters>,
                           sigma = <residual SD value>),
          family = gaussian)

As written this will generate a list with a single element which is a vector of responses.

  • what's written above is the maximal model (ignoring the possibility of interactions among the factors), which will almost certainly not work unless you have a huge number of subjects [in order to parameterize the 11x11 random effects covariance matrix]. You will probably need to boil it down; you could make the covariance matrix diagonal (using afex::mixed or using glmmTMB with diag() or something) or reduced-rank (using glmmTMB with rr())
  • it's quite common for people to use an intercept-only random effect (i.e. (1|subject), which would mean [if you were using sum-to-zero contrasts] that you would only allow the mean response to vary among individuals, not their responses to different factor levels. Technically, this model is incomplete if each subject experiences multiple levels of the different factors; in some cases this can lead to statistical problems (Schielzeth and Forstmeier 2009), but at the same time it's rarely practical to estimate all of the variation among subjects when there are many effects varying (as in this example), unless you're somehow regularizing the problem (e.g. with a factor-analytic/reduced-rank model or with informative priors in a Bayesian setting).

Schielzeth, Holger, and Wolfgang Forstmeier. “Conclusions beyond Support: Overconfident Estimates in Mixed Models.” Behavioral Ecology 20, no. 2 (March 1, 2009): 416–20. https://doi.org/10.1093/beheco/arn145.

$\endgroup$
1
  • $\begingroup$ Thank you, a follow up if that's okay? I thought the model would have (1|subject) for random intercept. But from the simulation code, it seems to suggest I'd need a random slopes per factor for each subject. I'm just wondering why that might be the case. $\endgroup$ Commented Feb 14 at 21:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.