1
$\begingroup$

Apologies as this has been posted before in similar topics but still I am trying to stretch my brain to understand it.

I have panel data, I am looking at both binary and count variables as outcomes and I am using random effect logistic and negative binomial regressions.

My two predictors are both discrete, one is smoking status (3 categories smoker non smoker and ex smoker) and the other is gender (binary) and I want to look at whether smoking cessation has an impact on reducing the effect of gender on selected outcomes.

First case: I have the two main effects and the interaction that are significant. Do have I to present in the table main effects and interaction or directly the RERI (in case of negative binomial regression)?

Second case: The two main effects are significant but not the interaction. Can I still say that the main effect of one is ..., the main effect of the other is... but smoking doesn't modify the effect of gender? Or I am not allowed to infer this in the specific case?

Third case: how to handle when the main effect (i.e. gender) and the interaction is significant but not the main effect of the other predictor (i.e. smoking)?

$\endgroup$
1
$\begingroup$

I'm going to limit my response to your random effects logit model because I have lots of experience with that - negative binomial not so much but I think my comments are applicable there as well.

I've found the best thing to do for ease of interpretation is to look at marginal effects, either average marginal effects or marginal effects at the mean. In general, marginal effects from a logistic model are much easier to understand that the more common odds ratios which reflect how much the baseline odds change for a 1 unit change in the predictor. Thus, if the baseline odds are a ratio of successes to failures then the odds ratio is a ratio of ratios - how confusing!

Marginal effects are linear or at least approximately linear for minute changes in X and are interpreted simply as the change in the probability of the outcome occurring.

Now, in the context of an interaction there is some research showing that significance tests of odds ratios are not a proper test to determine if the interaction is in fact significant. It may be at some levels of the covariates and not at others - the significance of the odds ratio for the interaction, aside from being confusing, is sort of a weighted average of the significance of the interaction across different levels of the covariates.

Assume X is the IV, Y the DV, and Z is the moderator. You should calculate the average marginal effect (or marginal effect at the mean if you prefer) for X across different levels of Z. If the line is horizontal then there is no interaction, however if it has a slope (and the confidence intervals for the point estimates are not overlapping) then there is an interaction. It's a supremely nice, intuitive, visual way to see what's going on and it's statistically proper (I think).

This is what is proposed by Ai and Norton. http://www.unc.edu/~enorton/AiNorton.pdf

If you use Stata this is easily implemented using the margins command and can be plotted visually using the marginsplot command.

Before you go worrying about what to say when X is not significant by the X*Z term is, I would correctly model the interaction. I suspect the confusion will resolve itself when you approach the issue with a more refined technique.

Here's a silly example to illustrate.

I used the standard Stata example dataset - the auto data - to model the odds of a car being foreign. The predictors are the price of the vehicle and the headroom. I test for an interaction between these variables.

logit of foreign on price headroom and an interaction term between them Note that the interaction is not significant.

Now we'll look closer at this with the margins command.

results from margins command

I modeled the average change in probability that results from a 1 dollar increase in price at various levels of headroom. The levels I choose 2, 2.5. 3, 3.5, and 4 correspond to the 10th, 25th, 50th, 75th, and 90th percentiles.

and now we can plot this with the command, marginsplot

and we get this: Average marginal effect of price across percentiles of headroom

We can see that increases in price increase the probability that a car will be foreign when the car has little headroom. When cars have more headroom, increases in price actually reduce the probability of them being foreign. In reality, the model probably has causality wrong - foreign probably causes price, price doesn't cause a car to be foreign... but then the outcome would be continuous and not suitable for logit regression and my example would be out the window (so please forgive the silly example).

Of course, the wide confidence intervals are problematic, at all point estimates they overlap so I would say that in no case is this interaction statistically significant. I wish I could whip up a better example where some of the confidence intervals overlapped and some did not so you could see how this approach is a bit more detailed and nuanced than the sort of global approach you get with the significance of the odds ratio for the interaction. Maybe someone else can do that with a simulation or has a dataset on hand where that is the case.

In any case and as you can probably tell, I am not a statistician by trade so caveat emptor.

$\endgroup$
  • $\begingroup$ Thank you! Can this approach be applied even when both the predictors are categorical? $\endgroup$ – user74520 Apr 25 '15 at 8:41
  • $\begingroup$ Absolutely, it would work in your case. You probably wouldn't need a graph or visual display to see what's going on though. Again, if using Stata, use the margins command to generate the predicted probability for each dyad. So you'd get a predicted probability for x1Z1, x1z2, x2z1, x2z2. These wouldn't be marginal effects but just the margins. In the neg bin model it would just be the predicted mean count for each combination of x and z. If you have other predictors in the model you could hold these as observed, at the mean, or at some other interesting value. $\endgroup$ – whauser Apr 26 '15 at 16:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.