# Negative coefficient in ordered logistic regression

Suppose we have the ordinal response $y:\{Bad,Neutral,Good\} \rightarrow \{1,2,3\}$ and a set of variables $X:=[x_1,x_2,x_3]$ that we think will explain $y$. We then do an ordered logistic regression of $X$ (design matrix) on $y$ (response).

Suppose the estimated coefficient of $x_1$, call it $\hat{\beta}_1$, in the ordered logistic regression is $-0.5$. How do I interpret the odds ratio (OR) of $e^{-0.5} = 0.607$?

Do I say "for a 1 unit increase in $x_1$, ceteris paribus, the odds of observing $Good$ are $0.607$ times the odds of observing $Bad\cup Neutral$, and for the same change in $x_1$, the odds of observing $Neutral \cup Good$ are $0.607$ times the odds of observing $Bad$"?

Can't find any examples of negative coefficient interpretation in my textbook or the Googlez.

-
Yes, that is correct. It is nearly identical to how you interpret positive coefficients. –  Peter Flom Sep 27 '12 at 10:08

You're on the right track, but always have a look at the documentation of the software you're using to see what model is actually fit. Assume a situation with a categorical dependent variable $Y$ with ordered categories $1, \ldots, g, \ldots, k$ and predictors $X_{1}, \ldots, X_{j}, \ldots, X_{p}$.

"In the wild", you can encounter three equivalent choices for writing the theoretical proportional-odds model with different implied parameter meanings:

1. $\text{logit}(p(Y \leqslant g)) = \ln \frac{p(Y \leqslant g)}{p(Y > g)} = \beta_{0_g} + \beta_{1} X_{1} + \dots + \beta_{p} X_{p} \quad(g = 1, \ldots, k-1)$
2. $\text{logit}(p(Y \leqslant g)) = \ln \frac{p(Y \leqslant g)}{p(Y > g)} = \beta_{0_g} - (\beta_{1} X_{1} + \dots + \beta_{p} X_{p}) \quad(g = 1, \ldots, k-1)$
3. $\text{logit}(p(Y \geqslant g)) = \ln \frac{p(Y \geqslant g)}{p(Y < g)} = \beta_{0_g} + \beta_{1} X_{1} + \dots + \beta_{p} X_{p} \quad(g = 2, \ldots, k)$

(Models 1 and 2 have the restriction that in the $k-1$ separate binary logistic regressions, the $\beta_{j}$ do not vary with $g$, and $\beta_{0_1} < \ldots < \beta_{0_g} < \ldots < \beta_{0_k-1}$, model 3 has the same restriction about the $\beta_{j}$, and requires that $\beta_{0_2} > \ldots > \beta_{0_g} > \ldots > \beta_{0_k}$)

• In model 1, a positive $\beta_{j}$ means that an increase in predictor $X_{j}$ is associated with increased odds for a lower category in $Y$.
• Model 1 is somewhat counterintuitive, therefore model 2 or 3 seem to be the preferred one in software. Here, a positive $\beta_{j}$ means that an increase in predictor $X_{j}$ is associated with increased odds for a higher category in $Y$.
• Models 1 and 2 lead to the same estimates for the $\beta_{0_g}$, but their estimates for the $\beta_{j}$ have opposite signs.
• Models 2 and 3 lead to the same estimates for the $\beta_{j}$, but their estimates for the $\beta_{0_g}$ have opposite signs.

Assuming your software uses model 2 or 3, you can say "with a 1 unit increase in $X_1$, ceteris paribus, the predicted odds of observing '$Y = \text{Good}$' vs. observing '$Y = \text{Neutral OR Bad}$' change by a factor of $e^{\hat{\beta}_{1}} = 0.607$.", and likewise "with a 1 unit increase in $X_1$, ceteris paribus, the predicted odds of observing '$Y = \text{Good OR Neutral}$' vs. observing '$Y = \text{Bad}$' change by a factor of $e^{\hat{\beta}_{1}} = 0.607$." Note that in the empirical case, we only have the predicted odds, not the actual ones.

Here are some additional illustrations for model 1 with $k = 4$ categories. First, the assumption of a linear model for the cumulative logits with proportional odds. Second, the implied probabilities of observing at most category $g$. The probabilities follow logistic functions with the same shape.

For the category probabilities themselves, the depicted model implies the following ordered functions:

P.S. To my knowledge, model 2 is used in SPSS as well as in R functions MASS::polr() and ordinal::clm(). Model 3 is used in R functions rms::lrm() and VGAM::vglm(). Unfortunately, I don't know about SAS and Stata.

-
@Harokitty The binary logistic regression model has no error term like the linear regression model. Note that we're modeling a probability, not the dependent variable itself. The assumption about an error distribution for $Y$ has to be specified separately, e.g., in R with glm(..., family=binomial). –  caracal Sep 27 '12 at 18:53
Do you have a reference that deals with the way of expressing specification #2 in your list of 3 alternatives? –  user14281 Oct 26 '12 at 3:06
@Harokitty It's briefly described in Agresti's "Analysis of Ordinal Categorical Data", section 3.2.2, p49, equation 3.8. Alternatively in Agresti's "Categorical Data Analysis", section 9.4, p323, equation 9.12. –  caracal Oct 26 '12 at 9:00
Hi, sorry to bother you, do you have a reference for the 3rd one? Agresti doesn't seem to talk about that. –  user14281 Nov 1 '12 at 4:01
@Jase Well, Agresti just uses $\text{logit}(Y > g)$ in the section linked above. For $\text{logit}(Y \geqslant g)$, see Harrell's "Regression Modeling Strategies", section 13.3.1, p333, eqn 13.4. –  caracal Nov 5 '12 at 9:08
show 1 more comment