8
$\begingroup$

I have been doing multinomial logistic regression analysis using SPSS 19. I have encountered the following problem when I run the analysis procedure:

"Unexpected singularities in the Hessian matrix are encountered. This indicates that either some predictor variables should be excluded or some categories should be merged."

A little background about my data used. I have four categorical predictors with two levels each, 1 or 2. The response variable in my model is a three-level categorical variable. I used the last level as the reference category. I tried to compare the coefficients of the intercept with that of the four predictors in the two logits so as to find which level of the response variable may cause this problem. The big differences in coefficients between the intercept and three of the predictors suggest that it might be the reference category that has the problem. However, I could not combine the levels of the response variable (which I'm not allowed for my research).

I have also tried to exclude the predictors one by one, but still got the same problem.

Could anyone please tell me what I should do to solve this problem?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ A first check would be to calculate the rank of your design matrix. If it's less than the number of columns, you probably need to combine and/or recode appropriately. $\endgroup$
    – cardinal
    Apr 29, 2011 at 16:47
  • $\begingroup$ Given that all variables are categorical, one alternative option is to use contingency table methods. I.e. you have a five way contingency table. This can be done using a poisson glm (log-linear model), which may be more stable (may not be though). Could also be a "separation problem" - your response can be perfectly predicted from the covariates - makes computers freak out when this happens because variance is zero. $\endgroup$
    – probabilityislogic
    Apr 30, 2011 at 11:01
  • $\begingroup$ Actually, combining levels of the response variable is a recommended way to approach problems in multinomial logistic regression. By combining the lower two levels and then the upper two levels you can approximate the multinomial results by means of two (simpler) logistic regressions. These logistic regressions and their diagnostics might indicate what's going wrong. $\endgroup$
    – whuber ♦
    Jun 29, 2011 at 15:59

1 Answer 1

Reset to default
4
$\begingroup$

I the key you may be looking for can be found on the UCLA website for Multinomial Logistic Regression where it states:

Perfect prediction: Perfect prediction means that only one value of a predictor variable is associated with only one value of the response variable. You can tell from the output of the regression coefficients that something is wrong. You can then do a two-way tabulation of the outcome variable with the problematic variable to confirm this and then rerun the model without the problematic variable.

I would recommend running a two-way table for each of the predictors (vs. the response) to determine if one level of the response occurs with only one level of your predictor.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.