2
$\begingroup$

I am performing (rare events) logistic regression analyses in R and want to test several categorical variables consisting of more than two classes. I understood that I can do this by using factor(). However, I am not sure what to do in case some, but not all categories are significant. Can I (1) just leave out the ones that are not significant, assuming their coefficients equal 0, or (2) do I have to include them all in the model, or (3) do I somehow have to recalibrate the logistic regression model with only the significant dummies?

Thanks in advance!

|cite|improve this question
$\endgroup$
  • $\begingroup$ I have been searching for an answer similar maybe you guys can help me. I am also trying to interpret categorical variables with more than two classes. Some are significant whilst other classes are not. what can I infer from the insignificant ones? does this mean the insignificant ones and the reference category are equally influence the dependent? For example: ETHNICITY Reference- Indian Other Asian - Sig = .273 Exp (b) = 1.123 African - Sig = .000 Exp (b) = .148 Many Thanks! $\endgroup$ – Kate Apr 20 '14 at 14:03
  • $\begingroup$ Welcome to the site, @Kate. This isn't an answer to the OP's question. Please only use the "Your Answer" field to provide answers. If you have your own question, click the [ASK QUESTION] at the top & ask it there, then we can help you properly. Since you are new here, you may want to take our tour, which contains information for new users. $\endgroup$ – gung♦ Apr 20 '14 at 14:49
4
$\begingroup$

The contrasts you mention are a function of the choice of reference cell, so are arbitrary. Removing dummy variables (combining categories) will ruin type I error, confidence interval coverage, and bias estimates. There is nothing wrong with having 'insignificant' effects in a model.

|cite|improve this answer
$\endgroup$
0
$\begingroup$

That depends on what you mean by leaving out factors that aren't significant, and how your logistic regression is coded. If one level of your factor is being treated as the reference group (it always has effect estimate = 0), and other levels of that factor are coded as dummy variables (1 for subjects in that level, 0 otherwise), removing the dummy variables for some other levels is equivalent to combining those levels with the reference group. In R, the default reference group is the value that comes first in alphabetical order, so think about whether that makes sense. This is exactly the same thing as recoding your categorical variable. However, if you're comparing several different codings (or, parametrizations) to see which one fits best, you need to account for this when reporting your results, because the probability of observing "significant" results by chance increases with the number of different models you try.

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ Nice explanation of dummy variables, but "combining those groups could make your model fit better overall" is a dangerous thing to say. It won't make the model fit the sample better: the out-of-sample fit could be better or worse. The OP hasn't indicated that prediction is the main goal of modelling, or that the full model is over-fitting; even if he had ,combining "non-significant" groups would be a poor model selection method: the default answer should be his option (2) - to leave things be. $\endgroup$ – Scortchi♦ Feb 17 '14 at 16:54
  • 2
    $\begingroup$ Everything said until the last sentence is correct and useful. The last remark is not correct, though, because it is based on an illusion: once you have included all the original levels in the model you have used all those degrees of freedom (DF) in the analysis and that needs to be respected in all subsequent analyses of the same data. Combining certain levels and then pretending this was originally intended creates an artificial apparent reduction in DF, which thereby can (in some cases) increase measures such as reduced $R^2$ and AIC. This, however, is just a way of deceiving yourself. $\endgroup$ – whuber♦ Feb 17 '14 at 17:24
  • $\begingroup$ Thank you for the edit: it is an improvement. When you bring that last thought to its logical conclusion, though, there seems to be no reason to combine categories at all: since the DF will remain the same in an honest comparison of fits, it is never the case that combining the categories can be an improvement. You should be concluding, then, that either the entire factor (with all its original categories) should remain in the model or else be completely removed. (This rule can sometimes be broken if other considerations, such as "simplicity" of the coefficients, are of concern.) $\endgroup$ – whuber♦ Feb 17 '14 at 22:46
  • $\begingroup$ I think simplicity of coefficients is almost always important. Even if it gives no improvement to the goodness-of-fit statistics, summarizing results more succinctly contributes to the ultimate goal of interpretation. $\endgroup$ – vafisher Feb 17 '14 at 22:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.