I have the following data set:

                 |       Scenario 1       |     Scenario 2         |
                 |Trial 1|Trial 2| Trial 3|Trial 1|Trial 2| Trial 3|
              S1 | ...
 Condition 1  S2 | ...
              S3 |
              S5 |
 Condition 2  S6 |
              S7 |

Thus the Trials are nested in the Scenarios and all of them are within subject. I am trying to run an ANOVA on this data set. Here is the model without defining that Scenarios (and Trials) are within subject.

 my_data.aov <- aov(value~Condition*Trial%in%Scenario,data=my_data) #works fine

But when I specify that these are within subject:

my_data.aov <- aov(value~Condition*Trial%in%Scenario+Error(Player/(Trial%in%Scenario)),data=my_data) 

I get the following error

In aov(value ~ Condition * Trial % in % Scenario + Error(Player/(Trial %in%  :
Error() model is singular

The closest set-up I could find was Split plot in R but there the subjects are nested inside each Trial not in each Condition.


Here is an example file in long format.

What about this approach?

If I treat each Trial as a sample, then I can collapse across Scenarios by averaging them, so I will have a simpler model, where each Subject's behavior is described per Scenario. And since I need to analyze the relationship of value~Condition*Scenario I can do so by defining the Error like Error(Subject/Scenario).

Will this approach invalidate my analysis?

  • $\begingroup$ What does the last sentence mean? Do all subjects get all the conditions? I'm afraid you don't have a Split-plot design and now you misspecified your model to have non-estimable parameters. $\endgroup$ – Horst Grünbusch May 12 '14 at 12:03
  • $\begingroup$ As the data description shows: S1, S2, S3, ... take only Condition 1 and S7, ... only Condition 2. But I want to compare the effects of Condition 1 with Condition 2. The reason why it fails (I think) is that each subject has the Scenario, which nests the Trial, which I could not clearly integrate in the model. $\endgroup$ – Pio May 12 '14 at 12:05
  • $\begingroup$ Now I see. Try Error(Player/(Trial*Subject)). The nesting is irrelevant to specify this error term. You have 3*2 measures for each subject and want to integrate all the possible covariances between these measures into your model. To this end, you don't need to specify that these covariances may be decomposable into one part from the trial and one from the scenario. The covariance would be the same if scenario and trial would be crossed. In fact this is the part that makes your covariance parameters non estimable. $\endgroup$ – Horst Grünbusch May 12 '14 at 12:41
  • $\begingroup$ You meant Error(Player/(Trial*Scenario))? $\endgroup$ – Pio May 12 '14 at 12:45
  • $\begingroup$ Ah, yes, of course, sorry! Does it work? $\endgroup$ – Horst Grünbusch May 12 '14 at 12:47

Without an example dataset (say from 6 subjects) it is hard to say for certain. You might try to use dput towards this end.

The error message Error() model is singular suggests to me that you have insufficient degrees of freedom to calculate your model. I think it is probably unlikely you meant to use %in% in this context, but maybe I'm wrong. Look at ?%in%. %in% produces a logical TRUE/FALSE vector of whether the Trial # matches the Scenario #. When you do estimate that subject effect then you only have two observations in the cell where Trial # matches the Scenario number and then you won't have enough observations per subject left over to also look at a Condition * Trial %in% Scenario interaction.



... and be sure it matches your expectations.

  • $\begingroup$ I am getting NA for mean and var. $\endgroup$ – Pio May 12 '14 at 15:12
  • $\begingroup$ var I expected to be NA because from a single variable you can't calculate variance. Do you have missing values in your dataset? summaryBy(value~Condition*Trial%in%Scenario,data=mydata,FUN=function(x) {any(is.na(x))}) $\endgroup$ – russellpierce May 12 '14 at 15:37
  • $\begingroup$ You would expect that the result would say "TRUE" if you are missing any values in that cell. $\endgroup$ – russellpierce May 12 '14 at 15:56
  • $\begingroup$ Yes, I have missing values. $\endgroup$ – Pio May 12 '14 at 17:17
  • 1
    $\begingroup$ Repeated measures ANOVA can not handle missing values. $\endgroup$ – russellpierce May 12 '14 at 17:29

I found a solution, even though it solves a my bigger problem, but as you could see from my question I went down a road, where I got stuck and I didn't really get an answer how to solve it. So I opted for the lme4 package and used the lmer function to model the relationship in my data.

BTW, the my approach where I suggest collapsing the data is a traditional approach in psychology, which I found out recently (after asking the question).

Finally the tutorial which saved the day for me (using lmer) is written by Bodo Winter, where he works on a dataset that almost matches mine -- even though it's not so obvious from the first time you read it.

In short my linear model looks like this:

 lmer(value~Condition*Scenario + (1+Scenario|Player) + (1|Scenario/Trial)

This perfectly models my experimental setup.

  • 1
    $\begingroup$ I'm perplexed as to why you selected this answer to a question that asks "what can cause". I answered what can cause. What can one do instead is a different question. Collapsing the data is traditional in psychology (and many other fields) because it yields data you can run an ANOVA on. The approach you decided on is the mixed model approach that Horst and I mentioned above (lmer is the R version of PROC MIXED). I suspect your next question might be "where are the p-values". Rather than get into why that is a bad idea, here is a link: cran.r-project.org/web/packages/afex/index.html $\endgroup$ – russellpierce May 20 '14 at 13:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.