I have the following experimental data and desperately lack expert advice:

  • Of a total of 60 Subjects, 20 are randomly assigned to one of three conditions (1X3) during treatment

  • Each subject performs six tasks in random order. This is because I did not have access to a sufficient number of subjects to have one subject perform one task per treatment.

  • There are three dependent variables. The dependent variables are based on task outcomes and are to be related to the treatment condition.

  • I have to control for the fact that task outcomes are not independent within subjects (referring to the 3 * 6 outcomes by each subject)

  • Subject specific variables should also be controlled as they supposedly influence task behavior (e.g. gender, product involvement)

My thesis advisor first suggested a MANCOVA model for analysis, but after reading into the topic, I don’t find it appropriate at all since it doesn’t account for the within subject error.

However, I want to do things correctly. I believe the correct model would be a multi-level model but I hardly find information how to render my design in a multi-level model. I have even trouble to formulate my design in a multi-level model, if I only have one dependent variable. I would greatly appreciate some help: What would my model look like, where can I read about my specific problem. Thanks.

  • $\begingroup$ It's not just a simple issue of adopting different or correct methods. Instead, a more important perspective is, what hypotheses or tests do you have in mind? Without knowing your research goal in terms of hypothesis testing, generic suggestions would be pointless. $\endgroup$ – bluepole Nov 28 '12 at 17:47
  • $\begingroup$ It would help if you could provide more info on how the dependent variables are related to each task. $\endgroup$ – Manoel Galdino Dec 5 '12 at 20:08
  • $\begingroup$ Well, the experiment tests trading in artificial market games after an false-feedback treatment (You know a lot about the market outcomes / You know little about the market outcomes / Control w/o feedback). The Hypotheses relate to how the image of ability in knowing the market outcomes impacts market behavior in 6 consecutive markets. H1: The better you think you are, the more turnover you will generate on the market; H2: The better you think you are, the earlier you will engage in trading; H3: The better you think you are, the more you will trade in the direction of your initial estimation $\endgroup$ – J.P. Luedtke Jan 20 '13 at 14:40

I don't know if I qualify as an expert, so take it with a grain of salt. Based on what you're said, the more complex model, which correct/control for (almost) everything would be something like this.

Denoting each dependent variable by $y_{1}, y_{2}$ and $y_{3}$, denoting treatment assigned to subject $i$ by $T_{i,1}, T_{i,2}$ and $T_{i,3}$ and the performance of each tasks by subject $i$ by $t_{i,1}, t_{i,2}, ..., t_{i,6}$, we have:

$[y_{i,1}, y_{i,2}, y_{i,3}]$ ~ $ N([\mu_{i,1}, \mu_{i,2}, \mu_{i,3}], \sigma^{2})$

Now, you have a multivariate normal and you can model the variance as being correlated among each marginal distribution.

Also, you can have a multivariate distribution on the means and model them as correlated as well.

Maybe you will need something like the inverse wishart distribution (I guess). I'd take a look at the book by Gelman and Hill about Multilevel Models, since they have some discussion about using inverse Wishart distributions as priors.

In any case, this seems to me to be a quite complex model. I'd go first with a simple model, not correcting for correlation among the dependent variables (i.e. run 3 separate analysis) neither correcting for the independent variables (the within error you mentioned). Then, I'd asses the fit of the model and, if there is clear room for improvement with a multilevel analysis, I'd move on to the simple multilevel model correcting for correlation withing subject. And then, if necessary, I"d move on to a full multilevel model like the one I outlined above.

I'm not quite sure if that's what you're looking for. Hope it helps a bit.

ps.: Gelman has some papers relating anova and multilevel that may help you.

| cite | improve this answer | |
  • $\begingroup$ Dear Manoel, thanks for your help. I believe you rendered my problem correctly. I started the analysis by neglecting dependent variable correlation and within-subject dependencies. I will now move on to check an increase in fit via HLM. However it is hard to find software that could account for the last step (dependent variable correlation and with-in indepedent variables). If anyone knows a package that can handle such analysis (without major programming skills required), I would be thankful for any hint. In any way many thanks to you! $\endgroup$ – J.P. Luedtke Jan 20 '13 at 14:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.