Between / Within-Subjects Analysis (with multiple dependent variables): Multi level expert around?

Question

I have the following experimental data and desperately lack expert advice:

• Of a total of 60 Subjects, 20 are randomly assigned to one of three conditions (1X3) during treatment

• Each subject performs six tasks in random order. This is because I did not have access to a sufficient number of subjects to have one subject perform one task per treatment.

• There are three dependent variables. The dependent variables are based on task outcomes and are to be related to the treatment condition.

• I have to control for the fact that task outcomes are not independent within subjects (referring to the 3 * 6 outcomes by each subject)

• Subject specific variables should also be controlled as they supposedly influence task behavior (e.g. gender, product involvement)

My thesis advisor first suggested a MANCOVA model for analysis, but after reading into the topic, I don’t find it appropriate at all since it doesn’t account for the within subject error.

However, I want to do things correctly. I believe the correct model would be a multi-level model but I hardly find information how to render my design in a multi-level model. I have even trouble to formulate my design in a multi-level model, if I only have one dependent variable. I would greatly appreciate some help: What would my model look like, where can I read about my specific problem. Thanks.

Answer 1

I don't know if I qualify as an expert, so take it with a grain of salt. Based on what you're said, the more complex model, which correct/control for (almost) everything would be something like this.

Denoting each dependent variable by $y_{1}, y_{2}$ and $y_{3}$, denoting treatment assigned to subject $i$ by $T_{i,1}, T_{i,2}$ and $T_{i,3}$ and the performance of each tasks by subject $i$ by $t_{i,1}, t_{i,2}, ..., t_{i,6}$, we have:

$[y_{i,1}, y_{i,2}, y_{i,3}]$ ~ $ N([\mu_{i,1}, \mu_{i,2}, \mu_{i,3}], \sigma^{2})$

Now, you have a multivariate normal and you can model the variance as being correlated among each marginal distribution.

Also, you can have a multivariate distribution on the means and model them as correlated as well.

Maybe you will need something like the inverse wishart distribution (I guess). I'd take a look at the book by Gelman and Hill about Multilevel Models, since they have some discussion about using inverse Wishart distributions as priors.

In any case, this seems to me to be a quite complex model. I'd go first with a simple model, not correcting for correlation among the dependent variables (i.e. run 3 separate analysis) neither correcting for the independent variables (the within error you mentioned). Then, I'd asses the fit of the model and, if there is clear room for improvement with a multilevel analysis, I'd move on to the simple multilevel model correcting for correlation withing subject. And then, if necessary, I"d move on to a full multilevel model like the one I outlined above.

I'm not quite sure if that's what you're looking for. Hope it helps a bit.

ps.: Gelman has some papers relating anova and multilevel that may help you.