- My study involves rating a series of 40 different statements from 1 (not profound) to 7 (very profound), therefore my DV is ordinal.
- There will are 2 groups of subjects who will do the same task, but one group will see the statements in an illegible font and the other group will see them in a legibile font. This is my between-subjects variable with two levels (font).
- From the 40 statements, 20 are profound and 20 are non-profound. This is my within-subjects variable with two levels (statement type).
- Lastly, statements will either have a high number or likes or a low number of likes, leading to my last within-subjects variable (metrics). Participant 1 will only see (for example) Statement A with high no. of likes, and Participant 2 will see Statement A with a low no. of likes, so that each participant does not see and rate the same statement twice.
Therefore, if I am right, there are three factors all with two levels: 1 between (font), 2 within (metrics, statement type).
My scope is to understand whether 1) there is a difference in rating scores between disfluent and fluent statements and 2) there is a difference in rating scores between high and low metric statements.
My first go-to analyses was to use a mixed-model ANOVA, to allow me to see the interactions between the statements. My choice was motivated by the fact that a previous similar study used an ANOVA. However, I understand that because the data is ordinal, one should be careful when using ordinal data in parametric tests, despite the ANOVA's robustness to departures from normality common in ordinal DVs.
I was advised of the possibility of random effects from participants as well as statements (since all statements are different). Therefore, I would imagine a Linear Mixed Effect Model (
lmer) is ideal. But, again, since the data is ordinal, I have also looked at the Cumulative Link Mixed Model (
clmm)which is a regression model for ordinal data.
Now, at this point, I am at a cross-roads: should I follow the analyses of previous literature, or take into account the random effects and therefore use a different type of analysis? Could anyone shed light on what is the most appropriate analysis for this design and question? Please do not hesitate to ask for more information.