3
$\begingroup$

I have been reading several CV posts on binary logistic regression but I am still confused for my current situation.

I am attempting to fit a binary logistic regression to a series of continuous and categorical variables in order to predict the mortality or the survival of animals (qual_status). Please see the str below:

> str(logit)
'data.frame':   136 obs. of  9 variables:
 $ id         : Factor w/ 135 levels "01001","01002",..: 26 27 28 29 30 31 32 33 34 35 ...
 $ gear       : Factor w/ 2 levels "j","sc": 2 1 1 2 1 2 1 2 2 1 ...
 $ depth      : num  146 163 179 190 194 172 172 175 240 214 ...
 $ length     : num  37 35 42 38 37 41 37 52 38 37 ...
 $ condition  : Factor w/ 4 levels "1","2","3","4": 1 1 4 1 4 2 2 1 2 1 ...
 $ in_water   : num  80 45 114 110 60 121 56 140 93 68 ...
 $ in_air     : num  60 136 128 136 165 118 220 90 177 240 ...
 $ delta_temp : num  8.5 8.4 8.3 8.5 8.5 8.6 8.6 8.7 8.7 8.7 ...
 $ qual_status: Factor w/ 2 levels "0","1": 1 1 2 1 2 1 2 1 1 1 ...

I have no issues fitting an the following additive binary logistic regression with the glm function:

glm(qual_status ~ gear + depth + length + condition + in_water + in_air + delta_temp, data = logit, family = binomial)

...but I am also interested at how these predictor variables interact with one another and possibly influence survival. However, when I attempt the following interactive binary logistic regression:

glm(qual_status ~ gear * depth * length * condition * in_water * in_air * delta_temp, data = logit, family = binomial)

I receive a warning message "glm.fit: fitted probabilities numerically 0 or 1 occurred", along with missing coefficients due to singularities (NA or <2e-16 *) when I use summary:

Call:
glm(formula = qual_status ~ gear * depth * length * condition * 
    in_water * in_air * delta_temp, family = binomial, data = logit)

Deviance Residuals: 
  [1]  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0
 [36]  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0
 [71]  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0
[106]  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0

Coefficients: (122 not defined because of singularities)
                                                            Estimate Std. Error    z value Pr(>|z|)    
(Intercept)                                                1.419e+30  5.400e+22   26274077   <2e-16 ***
gearsc                                                    -1.419e+30  5.400e+22  -26274077   <2e-16 ***
depth                                                      1.396e+28  4.040e+20   34539471   <2e-16 ***
length                                                     6.807e+28  1.836e+21   37079584   <2e-16 ***
condition2                                                -3.229e+30  8.559e+22  -37727993   <2e-16 ***
condition3                                                 1.747e+31  4.636e+23   37671986   <2e-16 ***
condition4                                                 9.007e+31  2.388e+24   37724167   <2e-16 ***
in_water                                                  -4.540e+28  1.263e+21  -35935748   <2e-16 ***
in_air                                                    -4.429e+28  1.182e+21  -37470809   <2e-16 ***
delta_temp                                                -1.778e+28  3.237e+21   -5492850   <2e-16 ***
gearsc:depth                                              -1.396e+28  4.040e+20  -34539471   <2e-16 ***
gearsc:length                                             -6.807e+28  1.836e+21  -37079584   <2e-16 ***
depth:length                                              -9.293e+26  2.450e+19  -37930778   <2e-16 ***
gearsc:condition2                                          1.348e+30  3.567e+22   37809001   <2e-16 ***
gearsc:condition3                                          2.816e+30  7.495e+22   37575317   <2e-16 ***
gearsc:condition4                                                 NA         NA         NA       NA

Fitting only the continuous variables to a binary logistic regression doesn't yield any warnings or singularities but the addition of the ordinal predictor variables causes issues. Along with avoiding these warnings, is there a function/package that can handle dummy variables (I believe that is what I am looking for) in logistic regressions in R?

Improve this question
$\endgroup$
1
  • $\begingroup$ Also, it seems like you're computing the interactions ignoring whether the predictors are quantitative or categorical. It doesn't really have a meaning to compute an interaction including a numerical feature as is (creating bins may be a solution). $\endgroup$
    – Aymen
    Aug 26, 2014 at 20:44

1 Answer 1

Reset to default
4
$\begingroup$

The short answer is that you're trying to model a regression with more slope parameters then you have observations.

For more meaningful results, try limiting the number of interaction terms you try to model. I mean, do you really want to have terms for every conceivable nonempty subset of your predictors? (For instance, do you really think that the gearsc:length:depth:in_water:condition4 coefficient is the key to your analysis?).

The result is that you'll have to do considerably more typing (since you'll be specifying the individual interaction terms instead of merely multiplying all the coefficients together in the formula). However, the upside is that you'll end up with stronger results.

Does this help?

Edit:

Now, if you really do want to model all those interaction terms, there are certain approaches you could take. For instance, you can check out this set of lecture notes for strategies for dealing with sparse matrices. However, I think the easiest thing here would be to manually add a handful of interaction terms since it's usually best to keep things as simple as they need to be (but no simpler).

Improve this answer
$\endgroup$
8
  • $\begingroup$ Thank you for your explanation. I understand what you are getting at with not listing every interaction term. Yet, I do not know which, if any interaction, have an influence on survival so this is very exploratory. My initial plan was to create a fully interactive model and use backward selection. $\endgroup$
    – ccapizzano
    Aug 10, 2014 at 22:27
  • $\begingroup$ @ccapizzano: Yeah, definitely include some interaction terms. For instance, you can add in all pairs of predictors (and maybe even take them three at a time) for your exploratory work--but all 7 will be overkill, especially since you don't have that much data to begin with. It'd be one thing if you had 10,000 or 100,000 observations--then you could get away with a bunch of interaction terms--but not with only 136. $\endgroup$
    – Steve S
    Aug 10, 2014 at 22:33
  • $\begingroup$ @ccapizzano: No problem--I know the feeling... Can you paste the glm code you entered? $\endgroup$
    – Steve S
    Aug 10, 2014 at 22:55
  • $\begingroup$ My apologies, I had an incorrect line of code (I will delete that previous comment). The output of the glm still responded with a warning of "glm.fit: fitted probabilities numerically 0 or 1 occurred". I suppose I included too many pair interactions. Would it be wise to fit the additive model as I described above, backward select for the most significant main effects, and work on interactions from there? $\endgroup$
    – ccapizzano
    Aug 10, 2014 at 22:57
  • $\begingroup$ Check out this question--it deals with exactly what's happening to you. $\endgroup$
    – Steve S
    Aug 10, 2014 at 23:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.