When and why should psychologists use ordered probit models rather than the general linear model?

Question

Psychologists often use the general linear model with ordinal independent/dependent variables (i.e. Likert scales to measure 'levels' of a psychological trait. For example, assigning numbers to the labels: "Strongly Disagree, Disagree, Neither, Agree, Strongly Agree")

Apparently, Dr. John K. Kruschke gave a talk explaining how this leads to errors and recommended ordered probit models instead: Analyzing ordinal data with metric models: What could possibly go wrong?

Unfortunately, I could not attend.

Could someone please explain when and why psychologists should use ordered probit models rather than the general linear model?

Many thanks :)

Answer 1

There is a published paper of the same name including the same author: TM Liddell and JK Kruschke, "Analyzing ordinal data with metric models: What could possibly go wrong?" Journal of Experimental Social Psychology 79 (2018), 328–348.

A major point is that if you, say, use integers to label levels of a Likert item or scale and then treat those labels as if they were numeric outcomes, you will get into trouble if the true distances between successive levels aren't necessarily the same. Nevertheless, of 68 papers they examined, all treated Likert scales as if they were numeric.

Furthermore, the authors note: "Aside from not having equal distances between levels, ordinal data also routinely violate the distributional assumptions of metric models." And even if the variances of ordinal values of groups being compared are equal, "the variances of the ordinal values do not reveal the underlying relation of variances of the latent values" that are of primary interest.

The authors recommend Bayesian approaches for analyzing such types of outcomes, but note that frequentist ordered-probit approaches can be used. There also are similar frequentist logistic-related ordinal approaches that avoid the problems of treating such outcome data as numeric.

Answer 2

Ordered-probit models are within the catalog of generalized linear models. The point is that they are a much better approach to ordinal data than pretending the data are metric (i.e., treating ordinal level '1' as numeric 1.0, and ordinal level '2' as numeric 2.0, ...). Ordered-probit models aren't necessarily the best or the correct model of ordinal data, but certainly better than pretending the data are metric and throwing them into least-squares regression. The article highlights cases when the two approaches yield very different results, especially when the the variances differ across groups.