I need some clarification in my understanding of what's going on under the hood of multinomial logistic regression (MLR).

I have a nominal (not ordinal!) dependent variable $Y$ that takes values $A$, $B$ and $C$ and single quantitative predictor, $X$, so I run MLR with $A$ set as reference level of $Y$. I get intercepts and regression coeficients for $B$ and $C$, say $b_{0B}$, $b_{1B}$, $b_{0C}$, $b_{1C}$.

All the sources I consulted (like this one) say that these are coeficients of following equations:

$$ log\left( \frac{P(Y=B)}{P(Y=A)}\right) = b_{0B}+b_{1B}X$$ $$ log\left( \frac{P(Y=C)}{P(Y=A)}\right) = b_{0C}+b_{1C}X$$

They (sources) say also that these estimates come from following procedure:

  1. Code $Y$ with dummies, say $Y_B$ that is $1$ if $Y=B$ and $0$ otherwise and $Y_C$ that is $1$ if $Y=C$ and $0$ otherwise.
  2. Find estimates for two "ordinary" logistic regressions (one for $Y_B$ and one for $Y_C$) at once.

My question is: since $Y_B$ is $1$ if $Y=B$ and $0$, shouldn't we interpret $b_{0B}$ and $b_{1B}$ as coeficients of

$$ log\left( \frac{P(Y_B=1)}{P(Y_B=0)}\right) = log\left( \frac{P(Y=B)}{P(Y=A | Y=C)}\right) = b_{0B}+b_{1B}X$$?

Second question (less important): Is the following procedure wrong? Why? What are it's drawbacks?

  1. Create 3 variables: $Y_A$ that is $1$ if $Y=A$ and $0$ otherwise, $Y_B$ that is $1$ if $Y=B$ and $0$ otherwise and $Y_C$ that is $1$ if $Y=C$ and $0$ otherwise
  2. Estimate three ordinary logistic regressions

I know that one of these three dummies is redundant, but thanks to it I could see how $X$ affects log odds of each posible value of $Y$ against any others (log odds of chosing $A$ against any other choice, log odds of chosing $B$ against any other choice, log odds of chosing $C$ against any other choice).

Thanks a lot for any thoughts on this!

  • $\begingroup$ 1. The first two displays are fit by excluding observations from the other categor(y/ies). So the first display can be fit with a logistic model on a subset of the data so that $Y \ne C$, the second on the subset with $Y \ne B$. 2. Your second approach does not account for the constraint that the predicted category probabilities must sum to 1 for every value of $X$ (unless the model is saturated). $\endgroup$ – AdamO Apr 18 '18 at 21:33
  • $\begingroup$ Thank you. Can you tell why can't we use all the observations and get my imterpretation with $P(Y=A) | P(Y=C)$ in denominator? $\endgroup$ – Łukasz Deryło Apr 19 '18 at 5:22
  • $\begingroup$ For $P(Y=A)/P(Y=C)$ to be an odds ratio, the denominator event has to be the converse of the numerator. That means the event $Y \ne A$ is the same as $Y = C$. This is only true when $Y \in A \cup C$. $\endgroup$ – AdamO Apr 19 '18 at 14:28
  • $\begingroup$ I was asking about $P(Y=A or Y=C)$ in denominator not about $P(Y=A) / P(Y=C)$ :) Sorry for typo $\endgroup$ – Łukasz Deryło Apr 19 '18 at 14:36
  • $\begingroup$ Okay, so suppose you fit two models: P(Y=B)/P(Y= A or C). then P(Y=A)/P(Y=C). You can use this to predict P(Y= A or B or C (forgive my sloppy notation. However, if you did say P(Y=A)/(P(Y=B or C) then P(Y=B)/P(Y=C) this will not give you the same predictions. You have introduced the model's dependence on the parametrization which is undesirable when equivalent inference could be obtained otherwise. $\endgroup$ – AdamO Apr 19 '18 at 14:38

I think your confusion is about censoring. One way to understand multinomial logit is by looking at it as competing risks. If you did the binomial regressions as $Y=B$ vs. $Y\ne B$, then you would allow $Y=C$ in the latter box.

In competing risks thinking you have a base category $A$, and two competing risks $B$ and $C$. If you go $B$ then you can't be $C$, thus you sensor out $C$ when estimating the binomial logit on $B$ from base $A$.

  • 1
    $\begingroup$ I'm not sure if it answers my questions (it certainly does not answer the second one). I gave my interpretation and asked if it is correct. You gave me another one... Why it is better than mine? $\endgroup$ – Łukasz Deryło Apr 18 '18 at 21:28
  • $\begingroup$ Your interpretation will not lead to an identifiable model $\endgroup$ – Aksakal Apr 18 '18 at 22:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.