Linear Mixed Effect Model vs. Mixed ANOVA vs. Ordinal Logistic Regression

Question

• My study involves rating a series of 40 different statements from 1 (not profound) to 7 (very profound), therefore my DV is ordinal.
  • There will are 2 groups of subjects who will do the same task, but one group will see the statements in an illegible font and the other group will see them in a legibile font. This is my between-subjects variable with two levels (font).
  • From the 40 statements, 20 are profound and 20 are non-profound. This is my within-subjects variable with two levels (statement type).
  • Lastly, statements will either have a high number or likes or a low number of likes, leading to my last within-subjects variable (metrics). Participant 1 will only see (for example) Statement A with high no. of likes, and Participant 2 will see Statement A with a low no. of likes, so that each participant does not see and rate the same statement twice.

Therefore, if I am right, there are three factors all with two levels: 1 between (font), 2 within (metrics, statement type).

My scope is to understand whether 1) there is a difference in rating scores between disfluent and fluent statements and 2) there is a difference in rating scores between high and low metric statements.

My first go-to analyses was to use a mixed-model ANOVA, to allow me to see the interactions between the statements. My choice was motivated by the fact that a previous similar study used an ANOVA. However, I understand that because the data is ordinal, one should be careful when using ordinal data in parametric tests, despite the ANOVA's robustness to departures from normality common in ordinal DVs.

I was advised of the possibility of random effects from participants as well as statements (since all statements are different). Therefore, I would imagine a Linear Mixed Effect Model (lmer) is ideal. But, again, since the data is ordinal, I have also looked at the Cumulative Link Mixed Model (clmm)which is a regression model for ordinal data.

Now, at this point, I am at a cross-roads: should I follow the analyses of previous literature, or take into account the random effects and therefore use a different type of analysis? Could anyone shed light on what is the most appropriate analysis for this design and question? Please do not hesitate to ask for more information.

Answer 1

I think you need a nonlinear multilevel model.

You need a multilevel model (which goes by various names including mixed model) because you have dependence among the error terms, violating an assumption of ANOVA/OLS regression.

You need an ordinal model because your DV is ordinal.

Looking at the documentation for clmm it appears to be what you need,

Answer 2

I would use a cumulative link function. With this you can still have your nested random effects and get standard deviation for your random effect variable.

What I usually do before ever fitting a cumulative link model is...

1. create your model using lme or lmer with all the necessary random effects. "AS you design, so shall you analyze." So if you have blocks or plots or reps, or samples, it should be in your model.
2. Then I create the model with my categorical response variable as numeric just to see what the model would look like.

So you can either treat your categorical response as numeric. Or if you treat it as categorical. You can...

1. Fit a cumulative link model with your random effects. In the summary output you will get estimates and standard deviation of random effects.