I have carried out a behavioural task on two groups of participants: a patient group and a healthy control group. The patient group was tested in two separate sessions, to include the effects of medication (on or off). I'd like to carry out a Bayesian ANOVA on this data to jointly assess the within-subject effect of medication (on vs off) and the across-subject effect of disease (patients vs controls). A new Bayesian ANOVA (BANOVA) package exists in R but from my knowledge it seems that I can't include the within- and across-subject data in the same model, since some participants (the healthy controls) only contribute to the across-subject factor. Does anybody have any ideas on how I might test this?


I would run a multilevel model, which can handle missing data without listwise deletion. Level 1 is observation, which contains your DV as well as the IV of measurement occasion, "on or off." Level 2 is the participant, which contains the IV of control vs. patient.

If you want to stay with the Bayesian route, I would consider looking into Andrew Gelman's work. He does a whole lot, but I would argue one of this most significant contributions and focuses is on Bayesian mixed (i.e., multilevel) models.

A Google search for "Bayesian mixed models Andrew Gelman" returned a number of helpful links: one, two, three, four.

I've re-read the question and it seems like you aren't asking about missing data, but instead that your design isn't fully-crossed. While I still recommend a Bayesian mixed model (and those links above), those methods cannot handle that entire missing cell. That is, having patients off medication, patients on medication, and control off medication (but not control on medication).

In that case, you could try two different tests: 1) Patients on vs. patients off. Then, in a separate analysis, 2) Patients on vs. control off.

I agree with @LiKao that, in retrospect, I would have measured controls at two time points to account for the effect of time passing on them, too.

I suppose you could somehow look at the difference between patients on vs. control off (there should be a small, or no, difference here) and patients off vs. control off (there should be a difference here). Then you could get the difference between those differences and bootstrap them or something to see if they are different from zero? It is an unusual design in my field that does experiments, but I'm sure there has to be some formal way of doing what I just described in a field where you often have these non-fully-crossed designs. It might help to first look for (a) how to handle non-fully-crossed designs (i.e., not a full factorial), and then (b) Bayesian approaches to these tests.

My answer is admittedly less-than-perfect, but hopefully it sets you (or someone else) in the right direction.

  • $\begingroup$ Thanks very much for your detailed comments. Interestingly, I have been using rstan for modelling the data according to a predefined model (it is a reinforcement learning paradigm, so the model involves reward prediction errors). However, what initiated my question was the desire to create a more simplified version of the modeling, to see how a single DV might be analyzed with the experiment designed as it was. The links you posted look very useful! The first one in particular looks like a good way to combine lme4 with rstan, and to use it to explain the DV with a few predictors. Thanks again! $\endgroup$ – b.mccoy Jul 11 '17 at 9:11

In general an ANOVA or MANOVA is only able to analyze data without any missing cells. One possible design, which is often used would be to also test the healthy controls at two-time points, in parallel with the patients. That way the clear effect of the medication (and not just of the time passing) can be identified.

An additional possibility would be to use a Bayesian Regression instead of a MANOVA. In case a MANOVA can be used, a regression and a MANOVA are completely the same, however, a regression is more flexible, being able to work with missing data, incomplete cells etc.

  • $\begingroup$ Thanks for your suggestions. The main reason for not testing the controls twice was that this was actually an fMRI experiment with a behavioral task, so the financial cost of having a perfect, fully-crossed design was quite high. A Bayesian regression seems like the best way to go. I had been using the lme4 package in R, but have now just discovered its Bayesian equivalent - blme. I will give this a try! $\endgroup$ – b.mccoy Jul 11 '17 at 8:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.