# Compare fit of ordinal to linear model

I´m doing multivariate modeling on a dataset where the dependent variable is ordinal-values running from 0-11. So far I´ve done general linear models and it runs alright. However, I´d also like to run it as an logistic ordinal regression.

• What is the recommended maximum number of categories in the dependent ordinal variable?
• Would I need to cut the variable into groups and reduce the number of possible categories?
• How can I compare the fit of an ordinal-outcome model to a linear regression model? Especially, if the ordinal variable has been cut into fewer groups?
-

It is a great idea to compare the two methods, because it often teaches us that the semiparametric proportional odds ordinal logistic model is a better fit than OLS. There are at least three direct ways to check this.

1. Compute the mean absolute difference between predicted and observed $Y$ for both methods. For the P.O. model you can estimate the mean or median $Y|X$.
2. Compute Kendall's $\tau$ between predicted and observed $Y$, separately for the two methods to measure predictive discrimination using a rank correlation measure.
3. If there were only one $X$ and it was categorical, you can look at the distribution of $Y$ stratified by $X$. OLS assumes [if you are computing $P$-values or confidence limits] that the normal inverse transformation of the empirical CDF of $Y$ when stratified by $X$ consists of parallel straight lines. The P.O. model assumes that the logit of the CDF consists of parallel (but not necessarily straight) lines.
-
Thx. That helped. But out of curiosity - how would I go about proving that a reduced number of categories in the dependent variable provides a superior fit, but at the expense of reduced precision? I.e. by trichotomizing (is that a word?) the dependent variable into 0-4=A,5-8=B,9-11=C it will be easier to create a good fit, but the precision of the prediction would be far less than using the entire 0-11 scale. Is there a measure that balances this tradeoff between fit and precision? – Misha May 5 '13 at 18:20
That would be tritomizing if the word existed [dichotomous = di (2) + tomous (cut)]. Collapsing categories is, on the average, a way to make the model less effective. The only time I have collapsed $Y$ is when there are two high-frequency categories that are out of order with repect to more than one $X$. I pool those categories. There are papers formally studying the pooling of categories. – Frank Harrell May 5 '13 at 18:32
I calculated the predicted Y for the ols model and also for the ordinal model (using rms-type="mean",codes=T). However, trying to calculate kendall $\tau$ using cor(obsY,predY,method="kendall) I´m unable to replicate the $\tau$ produced in the lrm output. What am I missing? – Misha May 30 '13 at 7:37
1. You can do ordinal logistic with 11 categories, as long as there is sufficient N in each category. The rule of thumb is that the category with the smallest N should have 10 per independent variable.

2. As far as I know, you can't directly compare the fits with some statistic that is output from both runs; what I often do is to make plots of the predicted values from each model: A scatter plot of one vs. the other, a box plot of the differences between them and sometimes a mean difference plot (this may be less familiar; it is a creation of John Tukey's. Plot the mean of the two on the x axis and the difference on the y axis)

Note: You do not need more than one observation per level of $Y$ if the ordering of $Y$ is used in the analysis (which is the case for the usual proportional odds model). You only need multiple observations if you are treating $Y$ categories as nominal (polytomous). The proportional odds model generalizes the Wilcoxon and Spearman tests which do not require ties in $Y$.

The only time you need to collapse $Y$ categories in ordinal logistic regression is when you are assessing the proportional odds assumption.

-
I do agree strongly that plotting difference vs mean is a good idea. It can be found in Neyman, J., Scott, E. L. and Shane, C. D. 1953. On the spatial distribution of galaxies a speciﬁc model. Astrophys. J. 117 92–133 and Oldham, P. D. 1962. A note on the analysis of repeated measurements of the same subjects. Journal of Chronic Diseases 15: 969–977. The oldest Tukey references I know are later. I suspect this is a case where the same smart idea could easily occur to many smart people, so that identifying first creator is tricky. – Nick Cox May 5 '13 at 12:27
OK, I have only heard of this as a "Tukey mean difference plot". Another case of "Stigler's law of eponymy" (no scientific discovery is named after its creator). The law is an example of itself, since Stigler credits Robert Merton with discovering it. – Peter Flom May 5 '13 at 12:31
Quite so. In medical statistics the name Bland-Altman plot is common, even standard. Martin Bland and Douglas Altman wrote an excellent series of papers explaining how to do method comparison, but they started much later. – Nick Cox May 5 '13 at 12:47
Thanks for the correction @FrankHarrell . – Peter Flom May 5 '13 at 15:13