Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

This is somewhat of a beginner's question, but how does one interpret an exp(B) result of 6.012 in a multinomial logistic regression model?

1) is it 6.012-1.0 = 5.012 = 5012% increase in risk?

or

2) 6.012/(1+6.012) = 0.857 = 85.7% increase in risk?

In case both alternatives are incorrect, can someone please mention the correct way?

Ive searched many resources on the internet and I get to these two alternatives, and I am not entirely sure which one is correct.

share|improve this question
add comment

2 Answers 2

It will take us a while to get there, but in summary, a one-unit change in the variable corresponding to B will multiply the relative risk of the outcome (compared to the base outcome) by 6.012.

It makes sense to express this as a "5012%" increase in relative risk, but that's a confusing and potentially misleading way to do it, because it suggests we should be thinking of the changes additively, when in fact the multinomial logistic model strongly encourages us to think multiplicatively. The modifier "relative" is essential, because a change in a variable is simultaneously changing the predicted probabilities of all outcomes, not just the one in question, so we have to compare probabilities (by means of ratios, not differences).

The rest of this reply develops the terminology and intuition needed to interpret these statements correctly.

Background

Let's start with ordinary logistic regression before moving on to the multinomial case.

For dependent (binary) variable $Y$ and independent variables $X_i$, the model is

$$\Pr[Y=1] = \frac{\exp(\beta_1 X_1 + \cdots + \beta_m X_m)}{1+\exp(\beta_1 X_1 + \cdots + \beta_m X_m)};$$

equivalently, assuming $0 \ne \Pr[Y=1] \ne 1$,

$$\log(\rho(X_1, \cdots, X_m)) = \log\frac{\Pr[Y=1]}{\Pr[Y=0]} = \beta_1 X_1 + \cdots + \beta_m X_m.$$

(This simply defines $\rho$, which is the odds as a function of the $X_i$.)

Without any loss of generality, index the $X_i$ so that $X_m$ is the variable and $\beta_m$ is the "B" in the question (so that $\exp(\beta_m)=6.012$). Fixing the values of $X_i, 1\le i\lt m$, and varying $X_m$ by a small amount $\delta$ yields

$$\log(\rho(\cdots, X_m+\delta)) - \log(\rho(\cdots, X_m)) = \beta_m \delta.$$

Thus, $\beta_m$ is the marginal change in log odds with respect to $X_m$.

To recover $\exp(\beta_m)$, evidently we must set $\delta=1$ and exponentiate the left hand side:

$$\eqalign{ \exp(\beta_m) &= \exp(\beta_m \times 1) \\ & = \exp( \log(\rho(\cdots, X_m+1)) - \log(\rho(\cdots, X_m))) \\ & = \frac{\rho(\cdots, X_m+1)}{\rho(\cdots, X_m)}. }$$

This exhibits $\exp(\beta_m)$ as the odds ratio for a one-unit increase in $X_m$. To develop an intuition for what this might mean, tabulate some values for a range of starting odds, rounding heavily to make the patterns stand out:

Starting odds  Ending odds  Starting Pr[Y=1]  Ending Pr[Y=1]
0.0001         0.0006       0.0001            0.0006
0.001          0.006        0.001             0.006
0.01           0.06         0.01              0.057
0.1            0.6          0.091             0.38
1.             6.           0.5               0.9
10.            60.          0.91              1.
100.           600.         0.99              1.

For really small odds, which correspond to really small probabilities, the effect of a one unit increase in $X_m$ is to multiply the odds or the probability by about 6.012. The multiplicative factor decreases as the odds (and probability) get larger, and has essentially vanished once the odds exceed 10 (the probability exceeds 0.9).

Ratio change in probability

As an additive change, there's not much of a difference between a probability of 0.0001 and 0.0006 (it's only 0.05%), nor is there much of a difference between 0.99 and 1. (only 1%). The largest additive effect occurs when the odds equal $1/\sqrt{6.012} \sim 0.408$, where the probability changes from 29% to 71%: a change of +42%.

Additive change in probability

We see, then, that if we express "risk" as an odds ratio, $\beta_m$ = "B" has a simple interpretation--the odds ratio equals $\beta_m$ for a unit increase in $X_m$--but when we express risk in some other fashion, such as a change in probabilities, the interpretation requires care to specify the starting probability.

Multinomial logistic regression

(This has been added as a later edit.)

Having recognized the value of using log odds to express chances, let's move on to the multinomial case. Now the dependent variable $Y$ can equal one of $k \ge 2$ categories, indexed by $i=1, 2, \ldots, k$. The relative probability that it is in category $i$ is

$$\Pr[Y_i] \sim \exp\left(\beta_1^{(i)} X_1 + \cdots + \beta_m^{(i)} X_m\right)$$

with parameters $\beta_j^{(i)}$ to be determined and writing $Y_i$ for $\Pr[Y=\text{category }i]$. As an abbreviation, let's write the right-hand expression as $p_i(X,\beta)$ or, where $X$ and $\beta$ are clear from the context, simply $p_i$. Normalizing to make all these relative probabilities sum to unity gives

$$\Pr[Y_i] =\frac{p_i(X,\beta)}{p_1(X,\beta) + \cdots + p_m(X,\beta)}.$$

(There is an ambiguity in the parameters: there are too many of them. Conventionally, one chooses a "base" category for comparison and forces all its coefficients to be zero. However, although this is necessary to report unique estimates of the betas, it is not needed to interpret the coefficients. To maintain the symmetry--that is, to avoid any artificial distinctions among the categories--let's not enforce any such constraint unless we have to.)

One way to interpret this model is to ask for the marginal rate of change of the log odds for any category (say category $i$) with respect to any one of the independent variables (say $X_j$). That is, when we change $X_j$ by a little bit, that induces a change in the log odds of $Y_i$. We are interested in the constant of proportionality relating these two changes. The Chain Rule of Calculus, together with a little algebra, tells us this rate of change is

$$\frac{\partial\ \text{log odds}(Y_i)}{\partial\ X_j} = \beta_j^{(i)} - \frac{\beta_j^{(1)}p_1 + \cdots + \beta_j^{(i-1)}p_{i-1} + \beta_j^{(i+1)}p_{i+1} +\cdots + \beta_j^{(k)}p_k}{p_1 + \cdots + p_{i-1} + p_{i+1} + \cdots + p_k}.$$

This has a relatively simple interpretation as the coefficient $\beta_j^{(i)}$ of $X_j$ in the formula for the chance that $Y$ is in category $i$ minus an "adjustment." The adjustment is the probability-weighted average of the coefficients of $X_j$ in all the other categories. The weights are computed using probabilities associated with the current values of the independent variables $X$. Thus, the marginal change in logs is not necessarily constant: it depends on the probabilities of all the other categories, not just the probability of the category in question (category $i$).

When there are just $k=2$ categories, this ought to reduce to ordinary logistic regression. Indeed, the probability weighting does nothing and (choosing $i=2$) gives simply the difference $\beta_j^{(2)} - \beta_j^{(1)}$. Letting category $i$ be the base case reduces this further to $\beta_j^{(2)}$, because we force $\beta_j^{(1)}=0$. Thus the new interpretation generalizes the old.

To interpret $\beta_j^{(i)}$ directly, then, we will isolate it on one side of the preceding formula, leading to:

The coefficient of $X_j$ for category $i$ equals the marginal change in the log odds of category $i$ with respect to the variable $X_j$, plus the probability-weighted average of the coefficients of all the other $X_{j'}$ for category $i$.

Another interpretation, albeit a little less direct, is afforded by (temporarily) setting category $i$ as the base case, thereby making $\beta_j^{(i)}=0$ for all the independent variables $X_j$:

The marginal rate of change in the log odds of the base case for variable $X_j$ is the negative of the probability-weighted average of its coefficients for all the other cases.

Actually using these interpretations typically requires extracting the betas and the probabilities from software output and performing the calculations as shown.

Finally, for the exponentiated coefficients, note that the ratio of probabilities among two outcomes (sometimes called the "relative risk" of $i$ compared to $i'$) is

$$\frac{Y_{i}}{Y_{i'}} = \frac{p_{i}(X,\beta)}{p_{i'}(X,\beta)}.$$

Let's increase $X_j$ by one unit to $X_j+1$. This multiplies $p_{i}$ by $\exp(\beta_j^{(i)})$ and $p_{i'}$ by $\exp(\beta_j^{(i')})$, whence the relative risk is multiplied by $\exp(\beta_j^{(i)}) / \exp(\beta_j^{(i')})$ = $\exp(\beta_j^{(i)}-\beta_j^{(i')})$. Taking category $i'$ to be the base case reduces this to $\exp(\beta_j^{(i)})$, leading us to say,

The exponentiated coefficient $\exp(\beta_j^{(i)})$ is the amount by which the relative risk $\Pr[Y = \text{category }i]/\Pr[Y = \text{base category}]$ is multiplied when variable $X_j$ is increased by one unit.

share|improve this answer
1  
Great explanations, but the OP explicitly asked for the multinomial model. I may be reading more into the question than the OP intended, and the explanation for the binary case may be adequate, but I would love to see this answer cover the general multinomial case too. Even though the parametrization is similar, the "log-odds" are in general with respect to an (arbitrary) reference category, and they are not really log-odds, and a unit change in $X_i$ results in a combined change of these "log-odds", and an increasing "log-odds" does not imply and increasing probability. –  NRH Oct 18 '11 at 21:09
    
@NRH That's an excellent point. I had somehow read "multivariate" instead of "multinomial." If I get a chance to return to this I will try to flesh out those details. Fortunately the same mode of analysis is effective in finding the correct interpretation. –  whuber Oct 18 '11 at 21:35
1  
+1 anyway. Hope you get a chance to return. –  NRH Oct 18 '11 at 21:58
    
@NRH Done. I welcome your suggestions (or anyone else's) about how to make the interpretation clearer, or for alternative interpretations. –  whuber Oct 19 '11 at 19:21
    
thanks for writing this down. The complete answer is a very good reference. –  NRH Oct 28 '11 at 7:17
add comment

Try considering this bit of explanation in addition to what @whuber has already written so well. If exp(B) = 6, then the odds ratio associated with an increase of 1 on the predictor in question is 6. In a multinomial context, by "odds ratio" we mean the ratio of these two quantities: a) the odds (not probability, but rather p/[1-p]) of a case taking the value of the dependent variable indicated in the output table in question, and b) the odds of a case taking the reference value of the dependent variable.

You seem to be looking to quantify the probability--rather than odds-- of a case being in one or the other category. To do this you would need to know what probabilities the case "started with" -- i.e., before we assumed the increase of 1 on the predictor in question. Ratios of probabilities will vary case by case, while the ratio of odds connected with an increase of 1 on the predictor stays the same.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.