2
$\begingroup$

I am trying to get my head around the question of what kind of model is most appropriate for the following data:

Every participant rated 14 written statements in terms of various aspects (e.g. credibility, writing style, and logic). All aspects were rated on a Likert scale from 1-7. The statements were presented randomly.

The dataset looks as follows in the long form (example with 4 statements instead of 14): enter image description here

I would like to explore what predicts the perceived credibility.

Predictors: writing_style, logic

Outcome/y: credibility

I am thinking of my data as having the individual at L2 and the statements (repeated measures/observations) at L1.

First, I was thinking that this is an instance of an LMM with repeated measures (as I repeatedly measure the same thing but it is measured for different items, i.e. statements, rather than for different points in time). However, most examples I encountered in literature or found online are longitudinal studies which does not apply to my data.

Question: I began to wonder whether my data needs to be understood as multivariate or whether it can still be analysed as LMM with repeated measures but perhaps something needs to be analysed differently as the sequence is not of interest?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

I am thinking of my data as having the individual at L2 and the statements (repeated measures/observations) at L1.

I don't see how statements are nested within individuals because each statement occurs within every individual.

Observations are repeated/nested/clustered within individuals, so observations on the same individual will be more alike one another than observations on another individual and hence not independent, and that's why we use a random intercept for ID. But observations are also nested within statements, that is, each observation "belongs" to a particular statement (as well as to an individual), and this creates further dependence among the observations for that statement, because observations on the same statement will be more similar to one another than those on another statement and hence the random intercept for statement. Statements are not nested within individuals because each statement does not "belong" to any one individual. So this is a 2-level model with crossed random effects (cross-classified)

To make this clearer, contrast your setup with a 3-level model, where repeated observations (L1) are made on pupils (L2) in schools (L3). Observations are nested within pupils and pupils are nested with schools. Each observation "belongs" to one pupil, and each pupil "belongs" to one school

You could model this as a (generalized) linear mixed model. Due to the nature of your outcome variable (7 point likert item) you should ideally fit a model that allows an ordinal outcome as well as crossed random effects, such as MCMCglmm or clmm in the ordinal package. With clmm this would look something like:

clmm(credibility ~ writing_style + logic + (1|ID) + (1|statement), data=mydata)
Improve this answer
$\endgroup$
4
  • $\begingroup$ Many thanks for your response. Maybe this is a case of me not using the correct terms. I suppose I meant that the observations are repeated within individuals and each observation can be linked to one of the 14 statements. So I thought that the observations are L1. What I do not fully understand is in what sense is this different from repeated measures where the observations (different time points) are at L1 and individuals are at L2? Would it be wrong to apply this idea to my data or is crossed random effects just a different approach to the same thing? $\endgroup$
    – grey
    Jun 17, 2016 at 14:53
  • 1
    $\begingroup$ Yes, you have repeated measures with observations (L1) nested within individuals (L2), but observations (L1) are also nested (separately) within statements (also L2), hence the cross-classified structure. I will expand on this by updating the answer in a moment. $\endgroup$
    – Robert Long
    Jun 17, 2016 at 16:01
  • $\begingroup$ Many thanks for your helpful elaboration on this. Just to check I get this right: If I were to have an experimental and a control group and the members of each of the two groups judge 14 statements (which are different for the EG and the CG), then subject and statement would be crossed within groups and the group variable would be one level higher? $\endgroup$
    – grey
    Jun 20, 2016 at 7:45
  • 1
    $\begingroup$ No, the treatment group would would be a fixed effect in the 2-level model. It wouldn't make sense to use it as a clustering variable because, a) there are only 2 levels and it's distribution would not follow anything resembling a normal distribution, and b) you may be interested in the "experimental effect" itself, rather than just controlling for it. $\endgroup$
    – Robert Long
    Jun 20, 2016 at 8:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.