6
$\begingroup$

Is it common practice (and adequate) to regroup two binary dependant variables into a single 4-level dependent variable to take advantage of the multinomial regression? For instance, say we have information on two related conditions (outcomes) A and B. A new 4-category variable would be defined such that:

category 1 = Neither conditions A nor B
category 2 = Condition A (only)
category 3 = Condition B (only)
category 4 = Both conditions A and B

This allows running a single multinomial regression instead of using two binary logistic models that include the same predictors.

Improve this question
$\endgroup$
2
  • $\begingroup$ Could you tell us what you're doing? Also, wouldn't such a model have 4 categories (what about neither A nor B)? $\endgroup$
    – Peter Flom
    Apr 10, 2012 at 15:43
  • $\begingroup$ @Peter this is strictly theoretical. I'm following a class right now and the prof uses MLR this way and I'm thinking that might be stretching a little the notion of a single outcome having "mutually exclusive categories". You're right for the 4th category, I'll do an edit for that. $\endgroup$
    – Dominic Comtois
    Apr 10, 2012 at 16:28

3 Answers 3

Reset to default
4
$\begingroup$

As @Riaz Rizvi suggests, this may not be a good idea.

Your scheme enforces a particular (and rather unlikely) covariance structure on the problem by flattening to a multinomial this way. Since you suspect, or at least wish to allow the possibility that the presence of A is informative of B, then you should be working with a bivariate probit. Working with two separate logistic models is not going to be able to represent this. The model is a regression with an explicit correlated bivariate latent variable generating the choice probabilities, as discussed briefly in the link and at greater length in good econometrics texts.

Improve this answer
$\endgroup$
11
  • 1
    $\begingroup$ I'm confused that in your link the bivariate probit is a multinomial model yet you say to use that is not a good idea. Did you mean it is a bad idea to use two separate logistic or probit models rather than a multinomial one? Or am I missing something? $\endgroup$
    – Momo
    Apr 10, 2012 at 22:25
  • $\begingroup$ The example on that page is a bivariate probit, but here's a more targeted exposition $\endgroup$
    – conjugateprior
    Apr 10, 2012 at 22:40
  • $\begingroup$ But well caught with the multinomial reference, I'll change that in the answer. $\endgroup$
    – conjugateprior
    Apr 10, 2012 at 22:44
  • $\begingroup$ It still sounds like you agree with his professor since you are advocating a joint model rather than separate models for each variable. $\endgroup$
    – Neil G
    Apr 10, 2012 at 22:49
  • 2
    $\begingroup$ Thanks for the response. Could you comment briefly on the consequences of using the multinomial method instead of a bivariate method in such a situation? (Would the estimates be biased in any way, or would their precision be under/overestimated?) $\endgroup$
    – Dominic Comtois
    Apr 11, 2012 at 15:51
1
$\begingroup$

A multinomial is perfectly fine in this situation, but it comes at two costs:

  • An explosion in the number of parameters. (If you were to combine $n$ binary variables like this, you would have $2^n$ parameters instead of the original $n$.)
  • The solution is harder to interpret if the original variables are actually independent. (If you would have had a simple relationship such that input variable $x$ implies dependent variable $y=1$, you would now have $x$ implies that the combined dependent variable takes on one of the many outcomes corresponding to $y=1$.)

The major advantage is that your model can use the additional parameters to encode distributions not possible in the original model.

Improve this answer
$\endgroup$
0
$\begingroup$

I don't think so. The multinomial distribution is derived from n independent variables, but your situation has two dependent variables. A multinomial regression is not applicable here.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Could you explain this a little bit further? I don't understand why you say that a multinomial distribution is derived from $n$ independent variables. Do you mean outcomes? The outcomes are definitely dependent since they're mutually exclusive. $\endgroup$
    – Neil G
    Apr 10, 2012 at 21:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.