12
$\begingroup$

What are the proper assumptions of Multinomial Logistic Regression? And what are the best tests to satisfy these assumptions using SPSS 18?

Improve this question
$\endgroup$
0

5 Answers 5

Reset to default
7
$\begingroup$

The key assumption in the MNL is that the errors are independently and identically distributed with a Gumbel extreme value distribution. The problem with testing this assumption is that it is made a priori. In standard regression you fit the least-squares curve, and measure the residual error. In a logit model, you assume that the error is already in the measurement of the point, and compute a likelihood function from that assumption.

An important assumption is that the sample be exogenous. If it is choice-based, there are corrections that need to be employed.

As far as assumptions on the model itself, Train describes three:

  1. Systematic, and non-random, taste variation.
  2. Proportional substitution among alternatives (a consequence of the IIA property).
  3. No serial correlation in the error term (panel data).

The first assumption you mostly just have to defend in the context of your problem. The third is largely the same, because the error terms are purely random.

The second is testable to a certain extent, however. If you specify a nested logit model, and it turns out that the inter-nest substitution pattern is entirely flexible ($\lambda = 1$) then you could have used the MNL model, and the IIA assumption is valid. But remember that the log-likelihood function for the nested logit model has local maxima, so you should make sure that you get $\lambda =1$ consistently.

As far as doing any of this in SPSS, I can't help you other than suggest you use the mlogit package in R instead. Sorry.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Also, the multinomial probit model gives comparable output with a different set of assumptions. So an MNP/MNL comparison can be valuable as well. $\endgroup$
    – gregmacfarlane
    Commented Jun 27, 2012 at 1:29
6
$\begingroup$

Assumptions:

  1. Outcome follows a categorical distribution (http://en.wikipedia.org/wiki/Categorical_distribution), which is linked to the covariates via a link function as in ordinary logistic regression
  2. Independence of observational units
  3. Linear relation between covariates and (link-transformed) expectation of the outcome

For assumption 1 to be fulfilled, the categories of your outcome need to be exclusive (non-overlapping) and exhaustive (covering all possible forms the outcome can take).

I don't really know if there are any proper statistical tests for assumption 2. For time-series data there is a test of autocorrelation called Durbin-Watson test. For other forms of correlated data, I think you would rather make that decision based on theoretical considerations (e.g., if your data come are derived from a cluster-sampling procedure, you would expect the data within clusters to be correlated).

As for assumption 3, in binary logistic regression you can plot binned residuals against estimated probabilities to see if the average residual is around 0 over the entire range of estimated probabilities. I suppose this can be generalized to multinomial regression by making (k-1) such plots instead for the different categories of an outcome with k categories.

EDIT:

Concerning alternative models: Assumption 1 is fairly straightforward to fulfill. You might run into trouble because you have to estimate a large number of parameters (k-1 different sets of intercepts and slope parameters). In such a case, you could for example collapse the outcome into a binary outcome and do a simple logistic regression.

If assumption 2 is violated, you could use a mixed model, which allows you to specify a dependence structure-

As for assumption 3, you could transform variables of which you suspect that they have a non-linear effect. A common transformation is for example to include age squared in health-related outcomes.

Improve this answer
$\endgroup$
1
  • $\begingroup$ To follow up on the first reply, you can use the Hausman-McFadden test for independence. $\endgroup$
    – Obe
    Commented Jan 23, 2018 at 19:25
3
$\begingroup$

One of the most important practical assumptions of multinomial logistic is that the number of observations in the smallest frequency category of $Y$ is large, for example 10 times the number of parameters from the right hand side of the model.

Improve this answer
$\endgroup$
2
  • $\begingroup$ It turns out that this is not always completely true. There has been some (very) recent work showing that consistent parameters can be estimated for alternatives that you never observe, provided you have some exogenous information on what the actual population frequency is. But this requires a different estimator, so in general you are correct. $\endgroup$
    – gregmacfarlane
    Commented Jun 27, 2012 at 16:15
  • 1
    $\begingroup$ Sounds like a Bayesian prior is being called for - can't disagree. But without external information the unconstrained multinomial logistic has an awful lot of parameters to estimate. $\endgroup$
    – Frank Harrell
    Commented Jun 28, 2012 at 2:27
2
$\begingroup$

@h_bauer has provided a good answer. Let me add a small complementary point: You can also test for a curvilinear relationship by adding curvilinear terms and performing a nested model test. For example, imagine you have $X_1$ as an explanatory variable, but you aren't sure whether the relationship between it and the link transformed expectation is a straight line. You could form a new model by adding $X_1^2$, and $X_1^3$, and then test to see if your new model fits better than your original model.

Another assumption of generalized linear models, like the multinomial logistic, is that the link function is correct. Strictly speaking, multinomial logistic regression uses only the logit link, but there are other multinomial model possibilities, such as the multinomial probit. Many people (somewhat sloppily) refer to any such model as "logistic" meaning only that the response variable is categorical, but the term really only properly refers to the logit link. For more on links, it may help you to read my answer here: Difference between logit and probit models.

Regarding addressing violations of these assumptions, it is mostly self-explanatory. If the observations are not independent, you can add the relevant fixed or random effects to make them so. If the relationship with a predictor is not linear, you can add transformed variables so that it is linear in the augmented predictor space. If the link is not appropriate, you can change it, etc. In general, multinomial logistic regression does not make very constraining assumptions.

Improve this answer
$\endgroup$
2
$\begingroup$

gmacfarlane has been very clear. But to be more precise, and I assume you perform a cross section analysis, the core assumption is the IIA (independence of irrelevant alternatives).
You can not force your data fit into the IIA assumption, you should test it and hope for it to be satisfied. SPSS could not handle the test until 2010 for sure. R of course does it, but it might me easier for you to migrate to Stata and implement the IIA tests provided by the mlogit postestimation commands.

If the IIA does not holds, mixed multinomial logit or nested logit are reasonable alternatives. The first one can be estimated within the gllamm, the second with the far more parsimonious nlogit command.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.