Multinomial logistic regression vs multiple binary logistic regression [duplicate]

Question

I was practising codes from the textbook, "Discovering Statistics using R".

I wonder if running two binary logistic regression can be another way of running one multinomial logistic regression.

I tried multinomial logistic regression with multinom, but getting the influential data points was difficult. I understand that multinomial logistic regression runs binary logistic regression for k-1 categories. So, I ran two binary logistic regressions, but I got slightly different coefficients from that of the multinomial.

I got the datasets from datasets, "Chat-Up Lines.dat"

I reorganized the original dataset into three datasets, each containing only one category and combined two categories of data (base and response). Then I applied glm to each data.

Here is the code.

noResponseOnly <- subset(chatData,Success=="No response/Walk Off")
getPhonenumOnly <- subset(chatData,Success=="Get Phone Number")
goHomeOnly <- subset(chatData,Success=="Go Home with Person")

PhoneBase <- rbind(noResponseOnly,getPhonenumOnly)
HomeBase <- rbind(noResponseOnly,goHomeOnly)

# two models using binary logit regression
phoneModel.2 <- glm(Success ~ Good_Mate + Funny + Gender + Sex + Gender*Sex + Funny*Gender,na.action = na.omit,data = PhoneBase,family = binomial())
summary(phoneModel.2)

homeModel.2 <- glm(Success ~ Good_Mate + Funny + Gender + Sex + Gender*Sex + Funny*Gender,na.action = na.omit,data = HomeBase,family = binomial())
summary(homeModel.2)

# the model using multinom()
chatModel <- multinom(Success~Good_Mate + Funny + Gender + Sex + Gender*Sex + Funny*Gender, data=chatData)
summary(chatModel,Wald=TRUE)

The outcomes were

> exp(phoneModel.2$coefficients)
           (Intercept)              Good_Mate                  Funny 
             0.1049014              1.1404107              1.1270241 
      Gender[T.Female]                    Sex   Gender[T.Female]:Sex 
             0.3001970              1.4450642              0.6448238 
Funny:Gender[T.Female] 
             1.6775339 
> exp(homeModel.2$coefficients)
           (Intercept)              Good_Mate                  Funny 
            0.03926519             1.11299595             1.33931116 
      Gender[T.Female]                    Sex   Gender[T.Female]:Sex 
            0.00105268             1.32339001             0.69539254 
Funny:Gender[T.Female] 
            3.58605635 
> exp(coef(chatModel))
                    (Intercept) Good_Mate    Funny Gender[T.Female]      Sex
Get Phone Number     0.16812119  1.140922 1.149566      0.192781497 1.318121
Go Home with Person  0.01375542  1.138829 1.375005      0.003602029 1.517841
                    Gender[T.Female]:Sex Funny:Gender[T.Female]
Get Phone Number               0.7058655               1.636318
Go Home with Person            0.6208551               3.229770

So there's a little difference between the models. I'm not sure what causes this discrepancy.

Answer 1

To put it simply, they are two different models and should not be expected to be equivalent. Specifically, the multinomial logistic regression places an additional constraint on the predicted probabilities, which is that they must be greater than 0. You might think that by only modeling $k-1$ categories using binary logistic regression and letting the probability for the $k$th category simply be 1 minus the sum of those probabilities, this would automatically be satisfied (i.e., binary logistic regression only produces probabilities greater than 0 and less than 1), but binary logistic regression might produce a probability of .4 for category 1 and .7 for category 2, which would imply that category 3 must receive a predicted probability of -.1, which is obviously impossible. In practice, this will not happen often, which is why the coefficients will be similar between the two approaches, but that additional constraint will nudge the multinomial coefficients toward ensuring all probabilities are between 0 and 1.

For a concrete example, consider the 8th unit in the dataset, who had a Success value of "No response/Walk Off" and therefore appears in both binary regressions. From the phone model, they receive a predicted probability for "Get Phone Number" of .847, and from the home model, they receive a probability for "Go Home with Person" of .696. If we compute the predicted probability of "No response/Walk Off" as 1 minus the sum of these two probabilities, we get a negative number, which obviously makes no sense. In contrast, from the multinomial model, we get predicted probabilities of .579, .308, and .113 for the three categories, respectively.

There are some good reasons to use multiple binary regressions, e.g., when the data are too large to fit in a single multinomial model, but in general, it is best to use multinomial regression when the problem calls for it.

If you're interested in a more technical answer using the likelihood score equations for the two approaches, let me know. This is a more intuitive answer but it does capture the essence of the issue.