Interpreting Ordinal Logistic Regression in R

Question

I am doing an ‘Ordinal Logistic Regression’ in R but I am struggling to interpret the output.

When looking at regression with continuous values I can understand the values and what they represent but when it becomes ordinal (categorical) data I don’t see how these numbers translate.

I will give an explanation of what I’ve done to help explain the issue:

• I have conducted a questionnaire study where I ask people to rate their response to a series of statements as either ‘Strongly Disagree’, ‘Disagree’, ‘Neutral’, ‘Agree’, or ‘Strongly Agree’. In this example ‘W1’ is one of the statements.

• I then want to have each statement in turn as the dependent variable and determine if any of the other factors about the respondents (age, gender, type of diagnosis etc.) significantly affects their response

• The columns of data are:

  Age: Ordinal, ’25-34’, ’35-44’, ’45-54’, ’55-64’, and ’65 and over’

  Gender: Categorical, ‘Male’, ‘Female’, ‘Nonbinary’, and ‘Other’

  Anxiety: Categorical, ‘Professional’, or ‘Self’

  Anxiety_Years: Ordinal, ‘1-4’, ‘5-9’, and ‘10+’

  W1: ‘Strongly Disagree’, ‘Disagree’, ‘Neutral’, ‘Agree’, or ‘Strongly Agree’.

How do I interpret the coefficient values and/or the p-values to determine which factors are significant?

For example, if this were using continuous data then you would say ‘for each increase of 1 for x, y increases by 14.542’.

What I’m struggling with is how I translate that idea into the categorical/ordinal data I have.

OUTPUT:

> m <- polr(W1 ~ Gender + Age + Anxiety + Anxiety_Years, data = data, Hess=TRUE)
> summary(m)
Call:
polr(formula = W1 ~ Gender + Age + Anxiety + Anxiety_Years, data = data, 
    Hess = TRUE)

Coefficients:
                            Value Std. Error  t value
GenderMale              -0.798444     0.3716 -2.14868
GenderNon-binary        -0.935156     1.4992 -0.62378
GenderOther              0.149260     1.6584  0.09000
GenderPrefer not to say -1.194734     1.2101 -0.98730
Age.L                   -0.470849     0.5495 -0.85686
Age.Q                   -0.338398     0.5027 -0.67312
Age.C                   -0.005273     0.4160 -0.01267
Age^4                   -0.280396     0.3549 -0.79017
Age^5                   -0.585081     0.3522 -1.66134
AnxietySelf             -0.412993     0.2953 -1.39845
Anxiety_Years.L         -0.028738     0.2815 -0.10209
Anxiety_Years.Q          0.110843     0.2895  0.38290

Intercepts:
                           Value   Std. Error t value
Strongly Disagree|Disagree -2.7227  0.3519    -7.7375
Disagree|Neutral           -2.1205  0.3138    -6.7572
Neutral|Agree              -1.2890  0.2797    -4.6092
Agree|Strongly Agree        0.7458  0.2625     2.8415

Residual Deviance: 493.9012 
AIC: 525.9012 
> 
> pval <- pnorm(abs(summary_table[, "t value"]),lower.tail = FALSE)* 2
> summary_table <- cbind(summary_table, "p value" = round(pval,3))
> summary_table
                                  Value Std. Error     t value p value p value
GenderMale                 -0.798443583  0.3715981 -2.14867502   0.032   0.032
GenderNon-binary           -0.935156341  1.4991854 -0.62377633   0.533   0.533
GenderOther                 0.149259763  1.6584121  0.09000161   0.928   0.928
GenderPrefer not to say    -1.194734361  1.2101003 -0.98730190   0.323   0.323
Age.L                      -0.470848554  0.5495048 -0.85685970   0.392   0.392
Age.Q                      -0.338397948  0.5027273 -0.67312434   0.501   0.501
Age.C                      -0.005272766  0.4160405 -0.01267369   0.990   0.990
Age^4                      -0.280395930  0.3548547 -0.79017116   0.429   0.429
Age^5                      -0.585080566  0.3521728 -1.66134492   0.097   0.097
AnxietySelf                -0.412992815  0.2953221 -1.39844851   0.162   0.162
Anxiety_Years.L            -0.028737810  0.2814997 -0.10208825   0.919   0.919
Anxiety_Years.Q             0.110843066  0.2894819  0.38290153   0.702   0.702
Strongly Disagree|Disagree -2.722745362  0.3518882 -7.73753063   0.000   0.000
Disagree|Neutral           -2.120546640  0.3138214 -6.75717664   0.000   0.000
Neutral|Agree              -1.289026602  0.2796661 -4.60916345   0.000   0.000
Agree|Strongly Agree        0.745845230  0.2624787  2.84154550   0.004   0.004
> 
> exp(cbind(OR = coef(m), confint(m)))
Waiting for profiling to be done...
                               OR      2.5 %    97.5 %
GenderMale              0.4500289 0.21607986  0.930578
GenderNon-binary        0.3925245 0.01385979 11.097570
GenderOther             1.1609745 0.03345565 40.330263
GenderPrefer not to say 0.3027844 0.02703941  3.925079
Age.L                   0.6244721 0.20716802  1.824266
Age.Q                   0.7129115 0.26034476  1.904788
Age.C                   0.9947411 0.43621455  2.248073
Age^4                   0.7554846 0.37508201  1.512078
Age^5                   0.5570610 0.27777257  1.107730
AnxietySelf             0.6616670 0.36979179  1.179271
Anxiety_Years.L         0.9716712 0.55805392  1.687981
Anxiety_Years.Q         1.1172196 0.63264524  1.972616
> 

EDIT

From Kat's suggestion, I have done a train and test, as well as a goodness to fit test.

Train and Test

The result was:

0.6140351

I have interpreted this to mean that the model misclassified 61% of the data and therefore isn't a very good model. I believe I have read somewhere that a misclassification rate of 10% or lower is what you would like to see.

#GET DATA 
data<-read.xlsx("Linear_Regression.xlsx","TESTLR")

##REORDER
data$Gender<-factor(data$Gender, levels = c("Male", "Female", "Non-binary", "Prefer not to say", "Other"))
data$Anxiety<-factor(data$Anxiety, levels = c("Self", "Professional"))
data$Age<-factor(data$Age, levels = c("18-24", "25-34", "35-44", "45-54", "55-64", "65 or above"),ordered=TRUE)
data$Anxiety_Years<-factor(data$Anxiety_Years, levels = c("1-4", "5-9", "10+"),ordered=TRUE)
data$W1<-factor(data$W1, levels = c("Strongly Disagree", "Disagree", "Neutral", "Agree", "Strongly Agree"),ordered=TRUE)

##PREPARE TRAINING AND TEST DATA
set.seed(100)
trainingRows <- sample(1:nrow(data), 0.7 * nrow(data))
trainingData <- data[trainingRows, ]
testData <- data[-trainingRows, ]

##BUILD ORDERED LOGICSTIC REGRESSION MODEL 
options(contrasts = c("contr.treatment", "contr.poly"))
polrMod <- polr(W1 ~ Gender + Age + Anxiety + Anxiety_Years, data = trainingData)
summary(polrMod)

##PREDICT 
predictedClass <- predict(polrMod, testData)  # predict the classes directly
head(predictedClass)
predictedScores <- predict(polrMod, testData, type="p")  # predict the probabilites
head(predictedScores)

##CONFUSION MATRIX AND MISCLASSIFICATION ERROR 
table(testData$W1, predictedClass)  # confusion matrix
mean(as.character(testData$W1) != as.character(predictedClass))  # misclassification error

Goodness to fit test

In addition to the code shown originally, I have also run:

lipsitz.test(m)

This does give me a result but also shows a warning. I have researched this warning but haven't been able to solve the issue.

I am now sure how to interpret these values to understand more about the model?

Lipsitz goodness of fit test for ordinal response models

data:  formula:  W1 ~ Gender + Age + Anxiety + Anxiety_Years
LR statistic = 13.996, df = 9, p-value = 0.1225

Warning message:
In lipsitz.test(m) :
  g >= n/5c. Running this test when g >= n/5c is not recommended.

Questions from Kat's explanation

1. For the gender, there are the options for Male, Female, Non-binary, Other, Prefer not to say. Your explanation of interpreting the Male is clear, and I can see how you would apply that to the other options on the list. What I'm unsure of is how do you comment about 'Female' as this isn't on the list?

2. When looking at the results, would you follow these steps:

a. Look at the p-values

b. Identify significant p-values (ones that are <0.01)

c. Calculate the 'odds less likely to agree' for the significant ones

d. Comment on these values you calculate

3. Is there a guide on the significance of 'odds less likely' and the confidence interval of when it is considered significant? In a statistics session I attended we were told about the guidance for interpreting p-values (see below) - is there something similar or 'odds less likely?

p-value < 0.001 very strong statistical evidence of an effect.

p-value < 0.01 strong statistical evidence of an effect.

p-value ≤ 0.05 some statistical evidence of an effect.

p-value >≈ 0.05 weak or limited statistical evidence for an effect

p-value > 0.1 no statistical evidence of an effect

Answer 1

I assume that you've already evaluated the assumptions necessary for ordinal regression. I see you've already determined the p-values. So it would help if you got the confidence intervals and the odds ratios.

Once you've determined these metrics, you'll be able to interpret the results more meaningfully. You'll find some great resources with UCLA (they offer R, in addition to numerous other stats software) or the Laerd website (all based on SPSS but still helpful interpretation content).

For example, here's one specific to ordinal regression in R. Here is more on interpreting the results. This is another useful resource, but this one didn't use R.

BTW, I see you're using base R. You'll probably find R a lot easier to work with if you use it with RStudio.

---

---

Update

**You didn't train and test nor use a goodness of fit test (at least that I can see); one or both of these are important to ensure your model suitably represents the data.

I didn't include an interpretation of the model in general, but you'll probably want to look at that, too.

Since I don't know what you asked in W1, I've created an arbitrary statement that will represent the W1:

Aliens are friendly.

Interpretation 1

For respondents that reported their gender as male, the odds of being less likely to agree that aliens are friendly is 2.22 (95% CI [1.07, 4.63]) times that of respondents that reported other genders, holding all other variables constant, t(1) = -2.15, p = .032.

(BTW—that's APA formatting.)

How did I get this information from what you gathered in your analysis?

First, look at the polarity of the $\beta$ value for gender: male (the coefficient value). Is it increasing? Is it decreasing?

The odds ratio is telling us more likely odds. You could write a more complicated interpretation and use that value. However, it is easier to find the inverse of these odds so that you know the odds of less likelihood.

The odds of less likely is $\frac{1}{OR}$ or $\frac{1}{0.4500289}$, which is approximately 2.22.

The high and low end of the confidence interval is going to flip as well. Instead of -> likely 0.21607986 -> even more likely .930578, you'll have $\frac{1}{0.930578}$ <- least likely $\frac{1}{0.21607986}$ <- less likely or 95% CI [1.07, 4.63].

Your independent variables don't do a great job of explaining your outcome, so these low confidence values aren't typically reported. I chose to go over whatever represents age^5 since the repeatability meets a 90% confidence level threshold. (If you're not aware, the p-value represents—the confidence in your result's repeatability.)

Interpretation 2

In this interpretation, I refer to age^5 as age group five.

For the respondents that reported their age in age group five, the odds of being less likely to agree that aliens are friendly is 1.795 times that of respondents in other age groups, holding all other variables constant.

Let me know if I've left something important out of my explanation or if you have any other questions.