0
$\begingroup$

I have data which consists of subjective ratings. The ratings are done by different judges with each subject being judged under different experimental conditions. Each subject undergoes each condition and every judge (i.e. for two judges and three conditions there are six ratings per subject).

I am interested in the effect of the specific conditions, but not the effect of the specific judges whom I consider as drawn from a large population of possible judges. That makes the condition a fixed factor and the judges a random factor.

1) I believe that both the condition and the judge are within-subjects factors because each subject is exposed to each combination of condition and judge. However, I am unsure because the same judges and same conditions are used for all subjects.

2) Which R function is best for this kind of data, and how would I specify the model? For example, I tried ezANOVA, but could not find how to specify the judges as a random factor.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I would go for package lme4.

If I understand correctly, mF below should be your model. It has condition as a fixed effect, while judge and subject as a random intercepts.

library(lme4)

# m0 = Null model without the fixed effects.
# m1 = Your model including the fixed effects.
# Set REML = FALSE for a meaningful model comparison

m0 <- lmer(rating ~ 1 + (1|judge) + (1|subject), data = data, REML = FALSE) 
mF <- lmer(rating ~ condition + (1|judge) + (1|subject), data = data, REML = FALSE) 

summary(m0)
summary(mF)

anova(m0, mF)

# If significant, fit the final model using REML = TRUE
# This is often recommended since it usually gives better estimates for random effects.

mF <- lmer(rating ~ condition + (1|judge) + (1|subject), data = data, REML = FALSE)

Edit: Per discussion below.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Why do you advocate for REML=FALSE..? REML is generally a good default. $\endgroup$
    – Tim
    Commented Jan 30, 2015 at 21:31
  • $\begingroup$ It is my understanding that REML isn't a meaningful way to compare models that differ in their fixed effects. The advice I've seen usually is to use ML for comparisons and REML for the final (best) model. $\endgroup$
    – foto51
    Commented Jan 31, 2015 at 2:58
  • $\begingroup$ Thanks, this was helpful. I also added a (1|subjects) term since it is all within-subjects. It seems to work correctly. $\endgroup$
    – DaleSpam
    Commented Jan 31, 2015 at 4:15
  • $\begingroup$ @foto51 ok, however it would be better if you included this description in your answer since we don't know if OP is already familiar with LMM's, and so not to give him or her impression that REML=FALSE is a good "default" approach. $\endgroup$
    – Tim
    Commented Jan 31, 2015 at 7:59
  • 1
    $\begingroup$ The point in that during REML one transform the response variable. The original model $y = X\beta + Z\gamma + \epsilon$ is multiplied by a matrix $K$ such that $KX = 0$; as a result you are fitting $Ky = K(Z\gamma + \epsilon)$. Clearly comparing the fit of two models having different response variables is nonsensical (as $K$ depends on $X$ and different fixed effects dictate different $X$ matrices). ie. Use ML for model comparison. :) (In fairness you won't usually get drastically different results if you use REML). $\endgroup$
    – usεr11852
    Commented Jan 31, 2015 at 13:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.