# Intuition behind multinomial logistic regression

I need some clarification in my understanding of what's going on under the hood of multinomial logistic regression (MLR).

I have a nominal (not ordinal!) dependent variable $Y$ that takes values $A$, $B$ and $C$ and single quantitative predictor, $X$, so I run MLR with $A$ set as reference level of $Y$. I get intercepts and regression coeficients for $B$ and $C$, say $b_{0B}$, $b_{1B}$, $b_{0C}$, $b_{1C}$.

All the sources I consulted (like this one) say that these are coeficients of following equations:

$$log\left( \frac{P(Y=B)}{P(Y=A)}\right) = b_{0B}+b_{1B}X$$ $$log\left( \frac{P(Y=C)}{P(Y=A)}\right) = b_{0C}+b_{1C}X$$

They (sources) say also that these estimates come from following procedure:

1. Code $Y$ with dummies, say $Y_B$ that is $1$ if $Y=B$ and $0$ otherwise and $Y_C$ that is $1$ if $Y=C$ and $0$ otherwise.
2. Find estimates for two "ordinary" logistic regressions (one for $Y_B$ and one for $Y_C$) at once.

My question is: since $Y_B$ is $1$ if $Y=B$ and $0$, shouldn't we interpret $b_{0B}$ and $b_{1B}$ as coeficients of

$$log\left( \frac{P(Y_B=1)}{P(Y_B=0)}\right) = log\left( \frac{P(Y=B)}{P(Y=A | Y=C)}\right) = b_{0B}+b_{1B}X$$?

Second question (less important): Is the following procedure wrong? Why? What are it's drawbacks?

1. Create 3 variables: $Y_A$ that is $1$ if $Y=A$ and $0$ otherwise, $Y_B$ that is $1$ if $Y=B$ and $0$ otherwise and $Y_C$ that is $1$ if $Y=C$ and $0$ otherwise
2. Estimate three ordinary logistic regressions

I know that one of these three dummies is redundant, but thanks to it I could see how $X$ affects log odds of each posible value of $Y$ against any others (log odds of chosing $A$ against any other choice, log odds of chosing $B$ against any other choice, log odds of chosing $C$ against any other choice).

Thanks a lot for any thoughts on this!

• 1. The first two displays are fit by excluding observations from the other categor(y/ies). So the first display can be fit with a logistic model on a subset of the data so that $Y \ne C$, the second on the subset with $Y \ne B$. 2. Your second approach does not account for the constraint that the predicted category probabilities must sum to 1 for every value of $X$ (unless the model is saturated). – AdamO Apr 18 '18 at 21:33
• Thank you. Can you tell why can't we use all the observations and get my imterpretation with $P(Y=A) | P(Y=C)$ in denominator? – Łukasz Deryło Apr 19 '18 at 5:22
• For $P(Y=A)/P(Y=C)$ to be an odds ratio, the denominator event has to be the converse of the numerator. That means the event $Y \ne A$ is the same as $Y = C$. This is only true when $Y \in A \cup C$. – AdamO Apr 19 '18 at 14:28
• I was asking about $P(Y=A or Y=C)$ in denominator not about $P(Y=A) / P(Y=C)$ :) Sorry for typo – Łukasz Deryło Apr 19 '18 at 14:36
• Okay, so suppose you fit two models: P(Y=B)/P(Y= A or C). then P(Y=A)/P(Y=C). You can use this to predict P(Y= A or B or C (forgive my sloppy notation. However, if you did say P(Y=A)/(P(Y=B or C) then P(Y=B)/P(Y=C) this will not give you the same predictions. You have introduced the model's dependence on the parametrization which is undesirable when equivalent inference could be obtained otherwise. – AdamO Apr 19 '18 at 14:38

## 1 Answer

I think your confusion is about censoring. One way to understand multinomial logit is by looking at it as competing risks. If you did the binomial regressions as $Y=B$ vs. $Y\ne B$, then you would allow $Y=C$ in the latter box.

In competing risks thinking you have a base category $A$, and two competing risks $B$ and $C$. If you go $B$ then you can't be $C$, thus you sensor out $C$ when estimating the binomial logit on $B$ from base $A$.

• I'm not sure if it answers my questions (it certainly does not answer the second one). I gave my interpretation and asked if it is correct. You gave me another one... Why it is better than mine? – Łukasz Deryło Apr 18 '18 at 21:28
• Your interpretation will not lead to an identifiable model – Aksakal Apr 18 '18 at 22:07