Ordinal Logistic Regression or mixed-model ANOVA?

Question

My experiment is a 2x2x2 mixed design, with one between-subjects factor and the other 2 IVs as within-subjects factors. All variables have two levels.

My DV comes in the form of profundity rating responses from 1 to 5 (1 = Not profound, 5 = Highly profound).

I wish to observe the interactions between the IVs and my first assumption is to use a mixed-model ANOVA. However, I've read that rating scale responses are not suitable for this measure as they violate the assumption of normality required to conduct an ANOVA. My understanding now is that an ordinal logistic regression is necessary for this.

However, my confusion lies in the fact that there are some papers that have conducted experiments similar to my study and have used rating responses in an ANOVA. Looking into this, I have seen claims saying that using 0 to 7 is acceptable for parametric tests, and as long as the first and last ratings are labelled and everything in between is blank, then it is possible to use an ANOVA.

Now I am confused as to which measure is suitable for my design and would like to ask this community whether they have heard of similar claims and what the most appropriate solution to this problem would be.

If any clarification is required, please do not hesitate to ask.

Thank you!

Answer 1

One key point, the conditional Normal distribution is what matters for ANOVA. In other words, if you had $Y_{ijk} = \mu + \beta_i + \beta_j + \beta_k (+\mbox{interactions?}) + \epsilon$ it's the distribution of the error term $\epsilon$ that matters.

As a statistical reviewer, I have had people try to fob me off with arguments about Normal approximations. I do know they might be accepted by non-statistical reviewers. But I can see all kind of good reasons for using POLR (proportional odds logistic regression). For example, what if the difference between ticking 5 and 4 is very different from the difference between 1 and 2. I know the effect sizes are in log odds ratios which is weird for most people, but they are very comparable linear models. Instead of $\mu$ above you have a vector of cutpoints, but then you get $\beta$ values which indicate the change in log odds for the difference between level 1 and level 2 in each of your factors. I know there are a few more assumptions to check, but I really think the advantages outweigh the time you have to invest in figuring it out.