2
$\begingroup$

I'm trying to predict the outcome "Decision" in the function of Age, Gender, Occupation, ....

The independent variable "Occupation" is known to be significant. But when I do the logistic model, each sub-group (modality) of it is not.

Should I regroup the levels having the same value of estimated coefficient? (which I guess doesn't make many sense because the levels are not statistically significant)

The variable Occupation has 74 different sub-groups.

And another problem is that when checking the multicollinearity, the function VIF in R doest work, it produces the NaN value, may be its due to the large number of sub-groups of Occupation.

Summary(Logistic Regression)

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ How many observations do you typically have per subgroup? Given how large those standard errors are, you may a problem with sparse data/complete separation of some levels of your categorical variable and your outcome. $\endgroup$
    – jsakaluk
    Jun 4, 2015 at 12:16
  • $\begingroup$ It varies from one sub-group to the others. The smallest number of observation is 1, and the biggest is ~1000. And you are perfectly right, I have the problem of complete separation for ~ 20 / 74 subgroups. And, there are many sub-groups that have the p very small. What should I do now? $\endgroup$
    – Metariat
    Jun 4, 2015 at 13:29
  • 2
    $\begingroup$ I would try to reduce the number of subgroups. If Occupation is a significant predictor of Decision, there is likely an underlying factor, such as professional stress level or something. You could try to group based on that. $\endgroup$
    – user3697176
    Jun 4, 2015 at 14:39
  • $\begingroup$ Even when I tried to reduce the number of subgroups, the p-value still doesn't decrease very much! What should I do? What is the problem here? I think about it alot but I dont manage to figure it out! $\endgroup$
    – Metariat
    Jun 4, 2015 at 14:47

2 Answers 2

Reset to default
1
$\begingroup$

Given your problem of sparse observations/complete separation, there are a few things you could try.

  1. As user3697176 suggests, reduce the number of subgroups you have by combining them in some meaningful way
  2. Reduce the number of predictors in your model (i.e., consider pruning some of your non-significant variables).
  3. Use exact tests (but this would be very computationally intensive)

and/or

  1. Use conditional maximum likelihood as the estimator of your model, which is better equipped to deal with the problem of sparse data (the survival package in R has this capability).

However, I would not consider the goal of this exercise to reduce particular p-values, per se. Rather, the problem with your model (at least as I see it) is that your standard errors for the levels of your categorical variable are clearly not being properly estimated. So, any of the above four solutions might fix your standard errors, but certain categories may remain as non-significantly associated with your outcome.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for the suggestions, thats very useful. And I have a question: Is it true when I say that the high value standard errors is due to the high level of p-value? So, reduce the s.e means reduce the p-value? $\endgroup$
    – Metariat
    Jun 4, 2015 at 15:28
  • 1
    $\begingroup$ Yes, but only if the parameter estimate stays the same, which might not be the case if/when you use a different estimator (e.g., conditional ML) $\endgroup$
    – jsakaluk
    Jun 4, 2015 at 15:30
0
$\begingroup$

Thank all of you for the contribution, I got the answer.

The problem here is the complete seperation of the reference class. Normally, R ranks the sub-classes alphabetically and chose the first sub-class as the reference class. The solution: change the reference class that doesn't make the outcome to perfectly separated as follow: 

new.variable <- relevel(old.variable, ref = "reference class that we chose")

then we can at the same time:

  1. Reduce the p-value
  2. Reduce the s.e of the estimated coefficients.

We still have some sub-classes that are not significant, but we can try to regroup them in a meaningful way, then we will have the good results.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.