Compare results of more than two ordinal models with different IVs I am using the ordinal package to create 4 different ordinal/cumulative link models. The dependent variable is a ranking (1, 2, 3, 4, or 5) and the independent variable is a different model score for each. My four models look like:
library(ordinal)
clm(opinion ~ linear.model.score)
clm(opinion ~ normalized.linear.model.score)
clm(opinion ~ log.model.score)
clm(opinion ~ normalized.log.model.score)

How can I compare all 4 models? I am trying to determine which version of my model's score makes the most consistent predictions, i.e. a higher model score consistently correlates to a higher opinion score. The model score doesn't have any bounds, though practically the linear score would be between 0 and ~100.
I am normalizing by the number of words in a test sentence, so the normalizing factor stays the same. In the log version, I am just taking the natural log of each model score. In addition, the model score is based on a handful of lexical factors. The opinion score just comes from an experiment.
 A: Thank you for all your clarifications, Adam. If you are willing to use R's rms package instead of the ordinal package, you can fit your models using the lrm() function. For example:
library("rms")

# proportional odds model
model1 <- lrm(opinion ~ linear.model.score, x = TRUE, y = TRUE)

# ordinal model summary
summary(model1)

# ordinal anova
anova(model1)

The rms package will produce discrimination indexes (e.g., $R^2$) and rank discrimination indexes for the model, (e.g., $C$) evaluated on the data at hand, which can be used as a basis for comparing predictive performance across models.
Because you will use your models for predictive purposes, you have to validate the models (preferably, using bootstrap) before you consider comparing them in terms of any of the indexes described above. As stated by E.W. Steyerberg in the chapter Overfitting and optimism in prediction models of the book Clinical Prediction Models (https://link.springer.com/chapter/10.1007/978-0-387-77244-8_5):
"A key threat to validity is overfitting, i.e. that the data under study are well described, but that predictions are not valid for new subjects. Overfitting causes optimism about a model's performance in new subjects."
In other words, models can perform well on the data at hand, but not necessarily on new data. Validation will produce optimism-corrected versions of all indexes.  You can compare these optimism-corrected versions for your chosen indexes. For example:
set.seed(132)

val1 <- validate(model1, method = "boot", B = 200) 

val1

will perform bootstrap validation of your first model using B = 200 bootstrap samples.
Discrimination is only one aspect of predictive performance; the other one is calibration.  Simple calibration statistics reported by the validate() function (i.e., calibration slope and intercept) do not address the issue of whether predicted values from the model are miscalibrated in a nonlinear - rather than linear - way. You can estimate and visualize an optimism-corrected calibration curve nonparametrically via the commands:
set.seed(132)

cal1 <- calibrate(model1, method = "boot", B = 200)

plot(cal1)

Dr. Frank Harrell's book on Regression Modelling Strategies (2nd Edition) will include all the information you need on model fitting, validation, diagnostics and presentation of model results using the rms package.
Addendum:
You could pick the model which provides the best performance across most of the indexes reported by rms.  This way, you don't have to choose a single index (since different indexes quantify different aspects of model performance and they all have limitations).
However,if you had to choose one optimism-corrected discrimination index, perhaps go for Somers' Dxy rank correlation?
The optimism-corrected calibration intercept should be close to 0 and the optimism-corrected calibration slope should be close to 1. If they are not, penalizing the model might help.
The optimism-corrected calibration curve should be 'close' to the 45-degree line (that is, the line with an intercept of 0 and a slope of 1).
