# Using multinomial logistic regression for multiple related outcomes

Is it common practice (and adequate) to regroup two binary dependant variables into a single 4-level dependent variable to take advantage of the multinomial regression? For instance, say we have information on two related conditions (outcomes) A and B. A new 4-category variable would be defined such that:

category 1 = Neither conditions A nor B
category 2 = Condition A (only)
category 3 = Condition B (only)
category 4 = Both conditions A and B


This allows running a single multinomial regression instead of using two binary logistic models that include the same predictors.

• Could you tell us what you're doing? Also, wouldn't such a model have 4 categories (what about neither A nor B)? – Peter Flom Apr 10 '12 at 15:43
• @Peter this is strictly theoretical. I'm following a class right now and the prof uses MLR this way and I'm thinking that might be stretching a little the notion of a single outcome having "mutually exclusive categories". You're right for the 4th category, I'll do an edit for that. – Dominic Comtois Apr 10 '12 at 16:28

As @Riaz Rizvi suggests, this may not be a good idea.

Your scheme enforces a particular (and rather unlikely) covariance structure on the problem by flattening to a multinomial this way. Since you suspect, or at least wish to allow the possibility that the presence of A is informative of B, then you should be working with a bivariate probit. Working with two separate logistic models is not going to be able to represent this. The model is a regression with an explicit correlated bivariate latent variable generating the choice probabilities, as discussed briefly in the link and at greater length in good econometrics texts.

• I'm confused that in your link the bivariate probit is a multinomial model yet you say to use that is not a good idea. Did you mean it is a bad idea to use two separate logistic or probit models rather than a multinomial one? Or am I missing something? – Momo Apr 10 '12 at 22:25
• The example on that page is a bivariate probit, but here's a more targeted exposition – conjugateprior Apr 10 '12 at 22:40
• But well caught with the multinomial reference, I'll change that in the answer. – conjugateprior Apr 10 '12 at 22:44
• It still sounds like you agree with his professor since you are advocating a joint model rather than separate models for each variable. – Neil G Apr 10 '12 at 22:49
• Thanks for the response. Could you comment briefly on the consequences of using the multinomial method instead of a bivariate method in such a situation? (Would the estimates be biased in any way, or would their precision be under/overestimated?) – Dominic Comtois Apr 11 '12 at 15:51

A multinomial is perfectly fine in this situation, but it comes at two costs:

• An explosion in the number of parameters. (If you were to combine $n$ binary variables like this, you would have $2^n$ parameters instead of the original $n$.)
• The solution is harder to interpret if the original variables are actually independent. (If you would have had a simple relationship such that input variable $x$ implies dependent variable $y=1$, you would now have $x$ implies that the combined dependent variable takes on one of the many outcomes corresponding to $y=1$.)

The major advantage is that your model can use the additional parameters to encode distributions not possible in the original model.

I don't think so. The multinomial distribution is derived from n independent variables, but your situation has two dependent variables. A multinomial regression is not applicable here.

• Could you explain this a little bit further? I don't understand why you say that a multinomial distribution is derived from $n$ independent variables. Do you mean outcomes? The outcomes are definitely dependent since they're mutually exclusive. – Neil G Apr 10 '12 at 21:49