I have a two by two design. Ideally I would run the whole thing within-subjects (all participants get all conditions) but the procedure is too long. I could run one factor between-subjects (each participants gets one condition) and one factor within-subjects(each participant gets both conditions), but then I have more power to see the effect for one factor vs. the other. I was thinking about running it such that one participant gets factor A within and another participant gets factor B within. That way I have to recruit fewer participants and I can see within subject and between subject effects for each factor. But I'm not sure how this would affect the analyses. Advice would be appreciated.

  • 1
    $\begingroup$ Very interested to hear others opinions. I don't think you'll necessarily end up with a great increase in the "efficiency" of the experiment in terms of participants recruited (which would end up with a part-way number between a fully-between and a fully-within design) to get the same amount of power/estimation accuracy -- Factor A will have more estimation accuracy in half the group, Factor B in the other half. The analysis for this might be excruciatingly complex, too -- perhaps some form of mixed (linear?) model, but I can't think how this would work exactly. $\endgroup$ – James Stanley Jan 15 '13 at 23:03
  • $\begingroup$ Thanks for your input. Would it be inappropriate to run the analyses as if all the data were between subjects? Clearly I'd be losing some power by not using the individual differences data but at least it would be less complicated. $\endgroup$ – Rebecca Jan 16 '13 at 2:16

Don't try to come up with a complicated analysis to deal with this, just do within and between analyses separately on the relevant subsets of your data. That way you will not compromise the interpretability of the outcome and you will be able to make sure that the result is both sensible and robust.

Depending on the nature of your data you could potentially combine the outcomes of the two arms of the experiment. For example, if you can generate likelihood functions for each arm then the combined function is simply their product.

  • $\begingroup$ I'm in agreement that complex analysis be avoided if possible. Essentially then your suggestion would be two separate experiments (in analysis practice, if not in data collection)? $\endgroup$ – James Stanley Jan 17 '13 at 0:31
  • $\begingroup$ Hi Michael, Thanks for your answer. I'm not sure I'm completely understanding. I would take the participants for whom Factor A was within-subjects and run a repeated measures analysis and the participants for whom Factor A was between-subjects and run a between-subjects analysis? $\endgroup$ – Rebecca Jan 17 '13 at 3:14
  • $\begingroup$ @Rebecca Yes, that sounds like what I have in mind. $\endgroup$ – Michael Lew Jan 17 '13 at 4:20
  • $\begingroup$ It seems like that is losing a lot of power. Wouldn't I be better off just running the whole thing between-subjects? Perhaps I should mention that I'm in a field in which participants are costly in time and effort, and that I'm looking at potentially small to medium sized effects. $\endgroup$ – Rebecca Jan 17 '13 at 21:23
  • $\begingroup$ @Rebecca It is not entirely clear whether one would loose power. You gain power from within subject designs when the within subject variability is small relative to the between subject variability. When there is little between subject variability the pairing might reduce the power because of the halving of sample size (although I'm not sure). If you want to design an experiment for maximal efficiency then you need ot have estimates of the various variances and effect sizes. $\endgroup$ – Michael Lew Jan 17 '13 at 21:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.