3
$\begingroup$

I'm trying to make a logistic regression model explaining whether a law passed last year has affected my dependent variable. My most important variable (an indicator variable for whether the law was in effect for a given observation with 1=law and 0=no law) has the wrong sign. Before the law came into effect, the dependent variable event happened 40% of the time (n=250), and after the law came into effect the event happened 56% of the time (n=40). However, the coefficient for the Law variable is negative and odds ratio below 1.

I am also using Date (or number of days after the first observation, as I coded it) as a variable. This is because frequency of the event was on an increasing trend over time, and I want to see if the increase in events after the law is due to the law itself or simply a continuation of the trend over time.

There are other control variables, but the sign on Law is only wrong when Date is included in the model. When I interact these two, the coefficient on Law is -50, the coefficient on the interaction term is 35, and the coefficient on Date is near 0. The law is significant with and without interaction, but not when Date is not included in the model.

Am I getting the wrong sign because these two variables (Law and Date) are collinear? If so, how would I go about figuring out what I want to know– whether the increase in events after the law was passed is due to the law or due to a continuation of the already-existing increasing trend?

Also- standardizing/normalizing Date has no or little effect.

Thank you so much for any advice or help, this has been a very confusing endeavor when I thought it was going to be quite simple.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

One way to look at this is that your probability of "success" (occurrence of your dependent variable) is a function of time. When you introduce the dummy at a fixed time point, you introduce a discontinuity into the time trend. In your model, this appears to be estimated as a drop at that time point, which I gather is not what you were hoping for.

On the other hand, one does not expect things to happen instantaneously, as your discontinuity model would suggest. It is somewhat encouraging from your perspective that your interaction is positive, because that suggests a steeper trend after than before.

Some caveats. First, time trends are not linear. They can be tricky to estimate because they generally do not follow nice function forms. Some kind of non- or semi-parametric smoothing trick might be preferred to estimate the time function. Second, you have one time function before the law, and another after the law. How to string those together will be tricky. A dummy variable introduces an instantaneous jump that seems inappropriate given that things take time to have an effect. Third, it seems that you really don't have enough data to do estimate everything as well as you would like to. Fourth, causality will be hard to claim in this context.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks so much! Do you have any more suggestions or resources about adding a non-parametric smoothing trick when using R? Does this mean something like adding Time^2? Also, you're correct about the steeper trend after the law, but should I be worried about how crazy the coefficients are with the interaction? I'm looking at the population, not a sample, so maybe since it's an explanatory model these coefficients don't matter? $\endgroup$
    – gus_mac
    Commented Aug 30, 2020 at 17:55
  • 1
    $\begingroup$ You might google "Nonparametric logistic regression." It's more flexible than Time^2, which is still too restrictive, but it's in the general direction. As far the coefficients being wild, that may be a function of sample size, and maybe it is also because you use "Year" as in the values 1999, 2000, 2001, ... rather than 1,2, 3, ... . You should graph the fitted probabilities as a function of time, along with the 0's and 1's, to understand the model better. The population "argument" really does not make sense here. $\endgroup$
    – BigBendRegion
    Commented Aug 30, 2020 at 19:14
  • $\begingroup$ Ah thank you, I was using years as a decimal (as in a certain observation could be 2.7234 years after the first observation), but your point about sample size makes sense. Unfortunately it's all the data there is. I'll see what I can do with non-parametric logistic regression. $\endgroup$
    – gus_mac
    Commented Aug 31, 2020 at 14:51
  • $\begingroup$ So I tried nonparametric regression and am still getting a negative coefficient. After graphing the smoothed effect of time on the dependent variable it still looks mostly linear, and I ran an anova comparing the parametric and nonparametric models which showed there's no/weak evidence for non-linearity for the time variable. Again, the sign is only wrong when I include the time variable, which is significant. Am I out of options? $\endgroup$
    – gus_mac
    Commented Sep 1, 2020 at 14:28
  • 1
    $\begingroup$ Sounds pretty interesting. Use the linear model then, with and without interaction, and draw the graph of the predicted probabilities as a function of year, including the effect of dummy in each model (two graphs). There will be a discontinuity at the point of time of the passage. It won't be in the direction you want, but it least it will give you a better idea what your data are telling you. It seems to me that your only hope is in the interaction model, but you probably do not have enough data to say too much. $\endgroup$
    – BigBendRegion
    Commented Sep 1, 2020 at 16:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.