I have a two by two design. Ideally I would run the whole thing within-subjects (all participants get all conditions) but the procedure is too long. I could run one factor between-subjects (each participants gets one condition) and one factor within-subjects(each participant gets both conditions), but then I have more power to see the effect for one factor vs. the other. I was thinking about running it such that one participant gets factor A within and another participant gets factor B within. That way I have to recruit fewer participants and I can see within subject and between subject effects for each factor. But I'm not sure how this would affect the analyses. Advice would be appreciated.
$\begingroup$
$\endgroup$
2
-
1$\begingroup$ Very interested to hear others opinions. I don't think you'll necessarily end up with a great increase in the "efficiency" of the experiment in terms of participants recruited (which would end up with a part-way number between a fully-between and a fully-within design) to get the same amount of power/estimation accuracy -- Factor A will have more estimation accuracy in half the group, Factor B in the other half. The analysis for this might be excruciatingly complex, too -- perhaps some form of mixed (linear?) model, but I can't think how this would work exactly. $\endgroup$– James StanleyCommented Jan 15, 2013 at 23:03
-
$\begingroup$ Thanks for your input. Would it be inappropriate to run the analyses as if all the data were between subjects? Clearly I'd be losing some power by not using the individual differences data but at least it would be less complicated. $\endgroup$– RebeccaCommented Jan 16, 2013 at 2:16
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
5
Don't try to come up with a complicated analysis to deal with this, just do within and between analyses separately on the relevant subsets of your data. That way you will not compromise the interpretability of the outcome and you will be able to make sure that the result is both sensible and robust.
Depending on the nature of your data you could potentially combine the outcomes of the two arms of the experiment. For example, if you can generate likelihood functions for each arm then the combined function is simply their product.
answered Jan 16, 2013 at 23:51
-
$\begingroup$ I'm in agreement that complex analysis be avoided if possible. Essentially then your suggestion would be two separate experiments (in analysis practice, if not in data collection)? $\endgroup$ Commented Jan 17, 2013 at 0:31
-
$\begingroup$ Hi Michael, Thanks for your answer. I'm not sure I'm completely understanding. I would take the participants for whom Factor A was within-subjects and run a repeated measures analysis and the participants for whom Factor A was between-subjects and run a between-subjects analysis? $\endgroup$– RebeccaCommented Jan 17, 2013 at 3:14
-
$\begingroup$ @Rebecca Yes, that sounds like what I have in mind. $\endgroup$ Commented Jan 17, 2013 at 4:20
-
$\begingroup$ It seems like that is losing a lot of power. Wouldn't I be better off just running the whole thing between-subjects? Perhaps I should mention that I'm in a field in which participants are costly in time and effort, and that I'm looking at potentially small to medium sized effects. $\endgroup$– RebeccaCommented Jan 17, 2013 at 21:23
-
$\begingroup$ @Rebecca It is not entirely clear whether one would loose power. You gain power from within subject designs when the within subject variability is small relative to the between subject variability. When there is little between subject variability the pairing might reduce the power because of the halving of sample size (although I'm not sure). If you want to design an experiment for maximal efficiency then you need ot have estimates of the various variances and effect sizes. $\endgroup$ Commented Jan 17, 2013 at 21:54