I have recently encountered a rather particular study design and wonder how it could be adequately described and if a multilevel model should be used to analyze it.

There are 3 factors in this fictional example:

  • Factor A, 2 levels (subject: math, world history)
  • Factor B, 2 levels (time of day: morning, afternoon)
  • Factor C, 3 levels (teacher: Mr Mustache, Miss Ponytail, Mrs Glasses)

This results in 12 combinations of subject, time of day, and teacher (see graphic). So, a1b1c1 would be "early math class with Mr Mustache" and a1b2c2 "early music class with Miss Ponytail". Those 12 classes are randomly combined into four study groups containing three classes (as represented by the color).

Students are randomly assigned to one of the four study groups, so each student participates in three classes. The dependent variable is students' satisfaction with class (measured for each of the 12 classes separately, say, on a scale from 1=not at all to 5=very much). So there are 3 satisfaction scores per student. The goal would be to estimate the effect of subject, time of day and teacher on class satisfaction.

For all the students we know their gender, age, and how much they like school in general.

enter image description here

  1. Students are nested within study groups. But that is the only thing I am confident saying.
  2. What about the three factors? Can you say that the factors A, B, and C are nested in something? Or crossed with something? And they are half within/half between subject - is it a 2x2x3 mixed (between/within subject) design?
  3. Which are fixed and which are random factors? The combinations (e.g. a1b1c1) are fixed, and always 3 combinations are in one group (fixed). But the assignment of the 3 combinations into one group is random (except for the last group, of course). And the allocation of students to one study group is also random.
  4. What about the control variables age, love-for-school and gender? Age and how much students like school in general can be entered as covariates, but would gender be entered as another factor creating a 2x2x2x3 mixed design?
  5. Is there another name for a "doubly mixed" design (between/within suject + random/fixed effects) or is it still just a "mixed design"?
  6. Would you analyze this kind of design with a multilevel model? Or are there any other ways to deal with the partially correlated data (so that uncontrolled student characteristics don't "ruin" the effects)?

This thread mentions "clustering on persons" - is that what this design does? This statement from another thread "I gave time both a fixed effect and a random effect" confuses me - so, factors can be both random and fixed at the same time? Yet another thread describes a similar design, but didn't get any responses.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.