I'm working on a project investigating the relationship between (let's say) a face's perceived masculinity and its perceived competence. There was a large number of face stimuli (80). Two completely separate groups of subjects rated the faces on each dimension: a given subject rated a random subset of the 80 faces on EITHER masculinity OR competence, never both. In other words, every subject rated multiple faces on a single dimension, and every face was rated by multiple subjects on both dimensions. (This was done to avoid interference effects and demand bias.) The specific face stimulus is considered a nuisance parameter here; face-specific effects are not really of interest.

A very simple modeling approach would be to treat face as the unit of analysis and characterize each face by its mean masculinity and its mean competence. This would result in a dataset with 1 row per face that would be easily amenable to linear regression, etc.

Obviously, this approach loses a lot of information, particularly failing to account for a subject random effect (such that, e.g., some subjects might generally consider faces more competent than other subjects). The problem invites a mixed-model with random intercepts or slopes by subject and face plus a fixed effect of masculinity, such as (in R shorthand):

competence ~ masculinity + (1|id) + (1|face)

However, I am not sure how this would work in light of the experimental design. I am basically having trouble envisioning how the fixed effect of masculinity will be estimable given that no observation contributes both a masculinity AND a competence rating.

Is this problem conducive to mixed-modeling? If so, how?


I tried fitting the mixed model specified as above using lmer to see what would happen. Not surprisingly, the model throws an error about incorrect grouping structure. This error goes away if I randomly simulate some of the missing-by-design such that there are now observations having both X and Y.

So, I am revising my question:

Is there a modeling approach for this situation that would retain more information than just modeling the face-specific means?


This is a rather interesting design and I hope that it is not preventing you from what you really want to measure.

Because subjects (id) do not have a measure of both masculinity and competence, you cannot, as you mention, associate them at the subject level. That is, you cannot test whether people who perceive a face as masculine also perceive it as more or less competent. Instead, you have to examine the relationship of these variables at the level of the face because each measure was taken for the different faces.

If you want to use mixed models, it seems that you should specify the model with a random intercept for id partially crossed with face. I am not so familiar with lmer, but it looks like you are specifying this correctly. This documentation would help with specification. Under this specification, you are coming to conclusions more along the lines of "faces that are perceived to be [more or less] masculine tend also to be perceived as [more or less] competent."

  • $\begingroup$ (+1) "faces that are perceived to be [more or less] masculine tend also to be perceived as [more or less] competent." -- Right, that's very much the intention. Thanks for the doc on multiple random effects, and I agree about the random slopes. I'm still wondering about whether mixed-modeling in general is a good idea, though. I know the model with churn through the ML and spit out a fixed effect of competence, but I just can't wrap my head conceptually around how a reasonable fixed effect would be estimated in this case. $\endgroup$ – half-pass May 25 '15 at 15:25
  • $\begingroup$ (On second thought, I think certain random slopes could make sense. For example, a random slope of masculinity by subject would model the possibility that certain subjects respond more strongly to masculinity than others.) $\endgroup$ – half-pass May 25 '15 at 17:06
  • $\begingroup$ I think what makes it so difficult to wrap one's mind around is that often fixed effects are estimated at the subject level where each person has a measurement of x and y. However, your "subject level" variable is face, while your actual participants are acting like detectors... it would be like measuring different chemicals by many different versions of the same machine in you lab, each with its own bias. So, I think the model makes sense, it is just counter intuitive. I really find this design fascinating! $\endgroup$ – Moose May 26 '15 at 8:35
  • $\begingroup$ And you are right about the random slopes - I think that could be a reasonable thing to test in the models. When I wrote the answer, I was thinking you would include a random slope on the "id" variable, which would not make sense. $\endgroup$ – Moose May 26 '15 at 8:37
  • $\begingroup$ Given this, I've edited out the comment on random slopes. $\endgroup$ – Moose May 26 '15 at 8:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.