2
$\begingroup$

I need some clarification in my understanding of what's going on under the hood of multinomial logistic regression (MLR).

I have a nominal (not ordinal!) dependent variable, $Y$, that takes values $A$, $B$ and $C$ and single quantitative predictor, $X$, so I run MLR with $A$ set as reference level of $Y$. I get intercepts and regression coefficients for $B$ and $C$, say $b_{0B}$, $b_{1B}$, $b_{0C}$, $b_{1C}$.

All the sources I consulted (like this one) say that these are coefficients of following equations:

\begin{align} \log\left( \frac{P(Y=B)}{P(Y=A)}\right) = b_{0B}+b_{1B}X \\[10pt] \log\left( \frac{P(Y=C)}{P(Y=A)}\right) = b_{0C}+b_{1C}X \end{align}

They (the sources) say also that these estimates come from following procedure:

  1. Code $Y$ with dummies, say $Y_B$ that is $1$ if $Y=B$ and $0$ otherwise and $Y_C$ that is $1$ if $Y=C$ and $0$ otherwise.
  2. Find estimates for two "ordinary" logistic regressions (one for $Y_B$ and one for $Y_C$) at once.

My question is: since $Y_B$ is $1$ if $Y=B$ and $0$, shouldn't we interpret $b_{0B}$ and $b_{1B}$ as coefficients of

$$ \log\left( \frac{P(Y_B=1)}{P(Y_B=0)}\right) = \log\left( \frac{P(Y=B)}{P(Y=A | Y=C)}\right) = b_{0B}+b_{1B}X \qquad ?$$

Second question (less important): Is the following procedure wrong? Why, what are its drawbacks?

  1. Create 3 variables: $Y_A$ that is $1$ if $Y=A$ and $0$ otherwise, $Y_B$ that is $1$ if $Y=B$ and $0$ otherwise and $Y_C$ that is $1$ if $Y=C$ and $0$ otherwise
  2. Estimate three ordinary logistic regressions

I know that one of these three dummies is redundant, but thanks to it I could see how $X$ affects log odds of each possible value of $Y$ against any others (log odds of choosing $A$ against any other choice, log odds of choosing $B$ against any other choice, log odds of choosing $C$ against any other choice).

$\endgroup$
7
  • $\begingroup$ 1. The first two displays are fit by excluding observations from the other categor(y/ies). So the first display can be fit with a logistic model on a subset of the data so that $Y \ne C$, the second on the subset with $Y \ne B$. 2. Your second approach does not account for the constraint that the predicted category probabilities must sum to 1 for every value of $X$ (unless the model is saturated). $\endgroup$
    – AdamO
    Commented Apr 18, 2018 at 21:33
  • $\begingroup$ Thank you. Can you tell why can't we use all the observations and get my imterpretation with $P(Y=A) | P(Y=C)$ in denominator? $\endgroup$ Commented Apr 19, 2018 at 5:22
  • $\begingroup$ For $P(Y=A)/P(Y=C)$ to be an odds ratio, the denominator event has to be the converse of the numerator. That means the event $Y \ne A$ is the same as $Y = C$. This is only true when $Y \in A \cup C$. $\endgroup$
    – AdamO
    Commented Apr 19, 2018 at 14:28
  • $\begingroup$ I was asking about $P(Y=A or Y=C)$ in denominator not about $P(Y=A) / P(Y=C)$ :) Sorry for typo $\endgroup$ Commented Apr 19, 2018 at 14:36
  • $\begingroup$ Okay, so suppose you fit two models: P(Y=B)/P(Y= A or C). then P(Y=A)/P(Y=C). You can use this to predict P(Y= A or B or C (forgive my sloppy notation. However, if you did say P(Y=A)/(P(Y=B or C) then P(Y=B)/P(Y=C) this will not give you the same predictions. You have introduced the model's dependence on the parametrization which is undesirable when equivalent inference could be obtained otherwise. $\endgroup$
    – AdamO
    Commented Apr 19, 2018 at 14:38

1 Answer 1

0
$\begingroup$

I think your confusion is about censoring. One way to understand multinomial logit is by looking at it as competing risks. If you did the binomial regressions as $Y=B$ vs. $Y\ne B$, then you would allow $Y=C$ in the latter box.

In competing risks thinking you have a base category $A$, and two competing risks $B$ and $C$. If you go $B$ then you can't be $C$, thus you sensor out $C$ when estimating the binomial logit on $B$ from base $A$.

$\endgroup$
2
  • 2
    $\begingroup$ I'm not sure if it answers my questions (it certainly does not answer the second one). I gave my interpretation and asked if it is correct. You gave me another one... Why it is better than mine? $\endgroup$ Commented Apr 18, 2018 at 21:28
  • $\begingroup$ Your interpretation will not lead to an identifiable model $\endgroup$
    – Aksakal
    Commented Apr 18, 2018 at 22:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.