Wrong sign in regression coefficient?

Question

I have been getting these extremely puzzling results in my logistic regression model. "New.Regs" is a dummy variable indicating whether or not an observation came after a certain law was passed.

The coefficients in the interaction model are extremely large. Without the interaction, the most important variable "New.Regs" is the wrong sign and has an odds ratio below 1 when in reality it has a positive effect. These results are not a symptom of complete separation (I have checked), and I also tried nonparametric smoothing for the Years variable (which is coded as a decimal, meaning an observation could for example be 2.7345 years after the first observation) but get the same results. Also, anova comparing models shows no evidence for non-linearity of Years. Also, n=250 before the law and n=40 after the law.

Any ideas? Some output is below. I know the graph isn't using the logit curve but you get the idea.

Model with interaction: glm(formula = Quantified ~ New.Regs * Years, family = binomial(link = "logit"), data = fws)

Coefficients:
                Estimate Std. Error z value Pr(>|z|)    
(Intercept)      -1.5728     0.3260  -4.824 1.40e-06 ***
New.Regs       -126.1103    44.7100  -2.821  0.00479 ** 
Years             0.2359     0.0547   4.314 1.61e-05 ***
New.Regs:Years   12.7242     4.5394   2.803  0.00506 ** 
AIC: 368.2

Model without interaction:

glm(formula = Quantified ~ New.Regs + Years, family = binomial(link = "logit"), data = fws)

Coefficients:
                                              Estimate Std. Error z value Pr(>|z|)    
(Intercept)                                   -2.94915    0.67265  -4.384 1.16e-05 ***
New.Regs                                      -1.25529    0.53753  -2.335  0.01953 *  
Years                                          0.38922    0.07473   5.209 1.90e-07 ***
---

    Null deviance: 399.19  on 290  degrees of freedom
Residual deviance: 303.64  on 272  degrees of freedom
AIC: 341.64

Model with only dummy variable, no time variable:

Call:
glm(formula = Quantified ~ New.Regs, 
    family = binomial(link = "logit"), data = fws)

Coefficients:
                                              Estimate Std. Error z value Pr(>|z|)    
(Intercept)                                   -0.98523    0.50487  -1.951  0.05100 .  
New.Regs                                       0.38726    0.41240   0.939  0.34771   

Residual deviance: 335.95  on 273  degrees of freedom
AIC: 371.95

Answer 1

The nonparametric smoothing, although often a good idea, is just a distraction here, at least with your bandwidth choices.

The results look less puzzling if you

• remember that a large negative constant in a logit model by itself just means a very small predicted probability

• and above all plot the predictions.

With the assumption that your time range in years is from 0 to about 11 I get these curves from your estimates:

I hope that helps. Easy to say, but hard to fix: you have only a short length of data after the law was passed. Despite the reported P-values I have to wonder about over-fitting here.