14
$\begingroup$

Suppose we have the ordinal response $y:\{\text{Bad, Neutral, Good}\} \rightarrow \{1,2,3\}$ and a set of variables $X:=[x_1,x_2,x_3]$ that we think will explain $y$. We then do an ordered logistic regression of $X$ (design matrix) on $y$ (response).

Suppose the estimated coefficient of $x_1$, call it $\hat{\beta}_1$, in the ordered logistic regression is $-0.5$. How do I interpret the odds ratio (OR) of $e^{-0.5} = 0.607$?

Do I say "for a 1 unit increase in $x_1$, ceteris paribus, the odds of observing $\text{Good}$ are $0.607$ times the odds of observing $\text{Bad}\cup \text{Neutral}$, and for the same change in $x_1$, the odds of observing $\text{Neutral} \cup \text{Good}$ are $0.607$ times the odds of observing $\text{Bad}$"?

I can't find any examples of negative coefficient interpretation in my textbook or Google.

$\endgroup$
  • 2
    $\begingroup$ Yes, that is correct. It is nearly identical to how you interpret positive coefficients. $\endgroup$ – Peter Flom Sep 27 '12 at 10:08
  • 2
    $\begingroup$ NB: usually we say "regress $y$ on $X$", not the other way around. $\endgroup$ – gung Mar 17 '15 at 15:06
23
$\begingroup$

You're on the right track, but always have a look at the documentation of the software you're using to see what model is actually fit. Assume a situation with a categorical dependent variable $Y$ with ordered categories $1, \ldots, g, \ldots, k$ and predictors $X_{1}, \ldots, X_{j}, \ldots, X_{p}$.

"In the wild", you can encounter three equivalent choices for writing the theoretical proportional-odds model with different implied parameter meanings:

  1. $\text{logit}(p(Y \leqslant g)) = \ln \frac{p(Y \leqslant g)}{p(Y > g)} = \beta_{0_g} + \beta_{1} X_{1} + \dots + \beta_{p} X_{p} \quad(g = 1, \ldots, k-1)$
  2. $\text{logit}(p(Y \leqslant g)) = \ln \frac{p(Y \leqslant g)}{p(Y > g)} = \beta_{0_g} - (\beta_{1} X_{1} + \dots + \beta_{p} X_{p}) \quad(g = 1, \ldots, k-1)$
  3. $\text{logit}(p(Y \geqslant g)) = \ln \frac{p(Y \geqslant g)}{p(Y < g)} = \beta_{0_g} + \beta_{1} X_{1} + \dots + \beta_{p} X_{p} \quad(g = 2, \ldots, k)$

(Models 1 and 2 have the restriction that in the $k-1$ separate binary logistic regressions, the $\beta_{j}$ do not vary with $g$, and $\beta_{0_1} < \ldots < \beta_{0_g} < \ldots < \beta_{0_k-1}$, model 3 has the same restriction about the $\beta_{j}$, and requires that $\beta_{0_2} > \ldots > \beta_{0_g} > \ldots > \beta_{0_k}$)

  • In model 1, a positive $\beta_{j}$ means that an increase in predictor $X_{j}$ is associated with increased odds for a lower category in $Y$.
  • Model 1 is somewhat counterintuitive, therefore model 2 or 3 seem to be the preferred one in software. Here, a positive $\beta_{j}$ means that an increase in predictor $X_{j}$ is associated with increased odds for a higher category in $Y$.
  • Models 1 and 2 lead to the same estimates for the $\beta_{0_g}$, but their estimates for the $\beta_{j}$ have opposite signs.
  • Models 2 and 3 lead to the same estimates for the $\beta_{j}$, but their estimates for the $\beta_{0_g}$ have opposite signs.

Assuming your software uses model 2 or 3, you can say "with a 1 unit increase in $X_1$, ceteris paribus, the predicted odds of observing '$Y = \text{Good}$' vs. observing '$Y = \text{Neutral OR Bad}$' change by a factor of $e^{\hat{\beta}_{1}} = 0.607$.", and likewise "with a 1 unit increase in $X_1$, ceteris paribus, the predicted odds of observing '$Y = \text{Good OR Neutral}$' vs. observing '$Y = \text{Bad}$' change by a factor of $e^{\hat{\beta}_{1}} = 0.607$." Note that in the empirical case, we only have the predicted odds, not the actual ones.

Here are some additional illustrations for model 1 with $k = 4$ categories. First, the assumption of a linear model for the cumulative logits with proportional odds. Second, the implied probabilities of observing at most category $g$. The probabilities follow logistic functions with the same shape. enter image description here

For the category probabilities themselves, the depicted model implies the following ordered functions: enter image description here

P.S. To my knowledge, model 2 is used in SPSS as well as in R functions MASS::polr() and ordinal::clm(). Model 3 is used in R functions rms::lrm() and VGAM::vglm(). Unfortunately, I don't know about SAS and Stata.

$\endgroup$
  • $\begingroup$ @Harokitty The binary logistic regression model has no error term like the linear regression model. Note that we're modeling a probability, not the dependent variable itself. The assumption about an error distribution for $Y$ has to be specified separately, e.g., in R with glm(..., family=binomial). $\endgroup$ – caracal Sep 27 '12 at 18:53
  • $\begingroup$ Do you have a reference that deals with the way of expressing specification #2 in your list of 3 alternatives? $\endgroup$ – user14281 Oct 26 '12 at 3:06
  • 1
    $\begingroup$ @Harokitty It's briefly described in Agresti's "Analysis of Ordinal Categorical Data", section 3.2.2, p49, equation 3.8. Alternatively in Agresti's "Categorical Data Analysis", section 9.4, p323, equation 9.12. $\endgroup$ – caracal Oct 26 '12 at 9:00
  • $\begingroup$ Hi, sorry to bother you, do you have a reference for the 3rd one? Agresti doesn't seem to talk about that. $\endgroup$ – user14281 Nov 1 '12 at 4:01
  • 2
    $\begingroup$ @Jase Well, Agresti just uses $\text{logit}(Y > g)$ in the section linked above. For $\text{logit}(Y \geqslant g)$, see Harrell's "Regression Modeling Strategies", section 13.3.1, p333, eqn 13.4. $\endgroup$ – caracal Nov 5 '12 at 9:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy