I am setting up an experiment where I would like to determine whether there exists a difference between the participants' average self reported mental effort in some condition A and condition B.

The participants will have to do an exercise and report their invested mental effort. Because of learning effects, a participant should not perform the same exercises in both condition A and B. However, I still would like to account for the fact that participants can have a different 'base level' of invested mental effort.

To deal with this, I want to (randomly) split the sample into two groups. Group 1 will do exercise set 1 in condition A, and exercise set 2 in condition B. Vice versa, group 2 will do exercise set 2 in condition A, and exercise set 1 in condition B. This means that every participant will be in both condition A and B, but with different exercises.

However, I am unsure what test would be appropriate to compare the means in condition A and condition B. Even though each participant is in both condition A and B, it is not truly a paired design, because the exercise set is different. On the other hand, it is also not a truly independent sample design, because the same people are in condition A and B.

I am only interested in the difference between condition A and B, so not necessarily in the effect of the exercise set (1 or 2). However, should I still perform e.g. a mixed ANOVA? If so, I am a bit unsure on how to define the factors. Or does there exist another test that is appropriate for this type of experiment design?

Any help would be appreciated!

  • $\begingroup$ You could estimate a subject effect for each subject (if you're prepared to assume no interaction between subject and condition), which should reduce the error variance if the variation between subjects is substantial; alternatively you may be better to use a mixed effects model. $\endgroup$
    – Glen_b
    Jan 17, 2019 at 17:29

1 Answer 1


You have a between participant factor (Group) and a within participant factor (condition A/B). Your mixed effects model will give you the effect of condition. The group effect is unlikely to correspond to any effect of scientific interest if I understand your problem. I would suggest ensuring that you have equal numbers in the two groups so that equal numbers of people in each condition do each exercise.

  • $\begingroup$ Thanks! Just to verify: the assigned group already implies which exercise set was assigned for each of the two conditions, so I do not have to take into account exercise set as a separate factor, right? Second, does it matter whether each entry consists of [participant, group, condition, exercise, mental effort] or [participant, group, condition, average mental effort over exercises]. As I am not interested in the effect of each question, I suppose the latter is sufficient, but I'm not sure. $\endgroup$
    – Hilde
    Jan 18, 2019 at 9:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.