10
$\begingroup$

I am trying to estimate a model of occupational choice with three choices. Are there any alternatives to using the multinomial logistic regression when handling such unordered categorical outcomes?

When dealing with binary dependent variables there seems to be several choices such as the LPM model as well as the binary probit and logit model. When dealing with unordered categorical variables the literature however keeps recommending the multinomial logit model without comparing it to alternatives.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Are you just asking if there are alternative link functions (other than the logit) available for the multinomial case? Or, are you asking about different types of models (such as cart models)? Or perhaps something else? (Side note, if the first, it might help you to read my answer here: difference-between-logit-and-probit-models, for general information about this issue, although it was written in a slightly different context.) $\endgroup$
    – gung - Reinstate Monica
    Commented Jul 21, 2013 at 16:14
  • $\begingroup$ Thank you very much for the comment. I will definitely read up on the link. I mainly wonder if there is any alternatives that uses ordinary linear regression (OLS) to handle unordered categorical outcomes. Do you know of any such alternatives? When it comes to binary outcomes there seems to be a whole discussion on whether to use OLS or binary logit/probit models. $\endgroup$
    – Thor
    Commented Jul 21, 2013 at 20:08
  • $\begingroup$ To a first approximation, OLS should never be used for binary outcomes. I'm sure there are, or could be, multinomial regression algorithms that use alternative link functions, but I don't know if major software supports them. $\endgroup$
    – gung - Reinstate Monica
    Commented Jul 21, 2013 at 20:13
  • 1
    $\begingroup$ Not sure what you mean by first approximation (sorry, I am a novice). But there seems to be some prominent econometricians arguing that using an LPM model functions as well as the logit model whan estimating binary outcomes. At least Angrist and Pischke do so in their book 'Mostly harmless econometrics' (2009). Do you have any tip on where I can read up on such alternative link functions? Again, thank you for your feedback! $\endgroup$
    – Thor
    Commented Jul 21, 2013 at 20:57
  • 1
    $\begingroup$ "To a first approximation... never... " means that 99% of the time you should not use OLS with a binary outcome. I am aware that there are some cases where it makes less of a difference & that some people disagree w/ the standard advice--that's why I didn't just say 'never' w/o the hedges. Unfortunately, I don't know of a good place to read up on using alternative link functions w/ multinomial regression. $\endgroup$
    – gung - Reinstate Monica
    Commented Jul 21, 2013 at 22:16

3 Answers 3

Reset to default
6
$\begingroup$

There is a variety of models available to model multinomial models.

I recommend Cameron & Trivedi Microeconometrics Using Stata for an easy and excellent introduction or take a look at the Imbens & Wooldridge Lecture Slides or here which are available online.

Widely used models include:

multinomial logistic regression or mlogit in Stata

multinomial conditional logit (allows to easily include not only individual-specific but also choice-specific predictors) or asclogit in Stata

nested logit (relax the independence from irrelevant alternatives assumption (IIA) by grouping/ranking choices in an hierarchical way) or nlogit in Stata

mixed logit (relaxes the IIA assumption by assuming e.g. normal distributed parameters) or mixlogit in Stata.

multinomial probit model (can further relax the IIA assumption but you should have choice-specific predictors available) mixed logit (relaxes the IIA assumption assuming e.g. normal distributed parameters), use asmprobit in Stata (mprobit does not allow to use choice-specific predictors but you should use them to relax the IIA asumption)

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ can I ask why mixed logit relaxes IIA ? it looks to me mixed logit is just bayesian logit in which the posterior distribution of $\beta$ is a mixture (number of modes == number of individuals) $\endgroup$
    – ElleryL
    Commented Oct 15, 2019 at 17:54
  • $\begingroup$ Yes of course, thanks, see Wikipedia for an explanation: https://en.wikipedia.org/wiki/Mixed_logit $\endgroup$
    – Arne Jonas Warnke
    Commented Oct 15, 2019 at 19:42
4
$\begingroup$

If you're wanting options quite different from a logistic regression, you could use a neural net. For example, R's nnet package has a multinom function. Or you could use a Random Forest (R's randomForest package, and others). And there are several other Machine Learning alternatives, though options like an SVM tend to not be well-calibrated which makes their outputs inferior -- in my opinion -- to a logistic regression.

[Actually, a logit is probably being used under the hood by the neurons in the neural net. So it's quite different, but not quite different at the same time.]

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ +1. Just to expand on a couple points...SVMs can be calibrated after training to produce good probabilities (e.g. using Platt scaling or isotonic regression, at the cost of an extra step). Neural nets with softmax outputs (and any type of nonlinear activation function in the hidden layers) can be thought of as simultaneously learning a nonlinear mapping into some feature space, and performing multinomial logistic regression in that feature space. $\endgroup$
    – user20160
    Commented Aug 16, 2017 at 12:30
  • 1
    $\begingroup$ @Wayne; I'm wondering that since multinomial logit requires IIA assumption; but about about neural network with softmax activation ? Does it also requires the same assumption? $\endgroup$
    – ElleryL
    Commented Oct 16, 2019 at 4:32
0
$\begingroup$

Also, think Neural Nets (with softmax activation), Decision Trees (or Random Forests) do not require the IIA assumption to be met considering the unreliability of these tests concerned with checking the IIA assumption. So this might be an advantage compared to the multinomial logistic if all we are concerned is only predictions.

Alternatively, multiple logistic models can be built for the K-1 categories with the Kth category as the reference. This also allows for different predictors to be plugged for each of the equations in contrast to the multinomial

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ This is rather a comment than an answer. We can either convert your answer in a comment or you can extend your answer. $\endgroup$
    – Ferdi
    Commented Aug 16, 2017 at 9:13
  • 1
    $\begingroup$ could you explain why Neural Nets (with softmax activation) do not require the IIA assumption. according to this, en.wikipedia.org/wiki/Luce%27s_choice_axiom I think Neural Nets (with softmax activation) has same restriction $\endgroup$
    – ElleryL
    Commented Oct 16, 2019 at 4:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.