4
$\begingroup$

EDIT 2

In reply to one of the commenters, here is the 2x2x2 table. Y = 1 : Y = 0.

X = 1 X = 0
M = 1 9 : 73 3 : 29
M = 0 34 : 245 1,214 : 21,204

EDIT 1

In my attempts to make things simpler when posting this question, I've made things more complicated. Here are the actual values.

X = 1 X = 0
Y = 1 43 1,217
Y = 0 318 21,233
X = 1 X = 0
M = 1 100 49
M = 0 331 26,748
M = 1 M = 0
Y = 1 12 1,248
Y = 0 102 21,449

$Y = X$ -> OR 2.1 (95% CI: 1.5, 2.9)

$Y = M$ -> OR 2.6 (95% CI: 1.5, 4.5)

$Y = X + M$ -> OR_X 1.9 (95% CI: 1.3, 2.7); OR_M 1.7 (95% CI 0.9, 3.1)

$Y = X * M$ -> OR_X 2.2 (95% CI: 1.5, 3.1); OR_M 4.4 (95% CI 1.8, 9.4); OR_interact 0.2 (0.06, 0.7)

For only subgroup of only $X = 1$, $Y = M$ -> OR 0.9 (95% CI: 0.3, 1.9)

For only subgroup of only $X = 0$, $Y = M$ -> OR 4.4 (95% CI: 1.8, 9.4)

These logistic models were all weighted using survey sampling weights.


I have a group of about $27,000$ people, $431$ of whom had an exposure $X$ (binary), the rest did not. $149$ used a certain medication $M$ (binary), most of whom ($100$) were people who also had exposure $X$. About $1,260$ people had outcome $Y$ (binary), including $43$ people who had $X = 1$ and $1,217$% people who had $X = 0$. $4,347$ datapoints for $Y$ were NA.

I'm fitting a logistic regression $Y = X + M$. $X$ has a significant OR ($1.9$) at an alpha of $.05$, but $M$ is insignificant ($1.7$). When I test $Y = X * M$, $X$ has a higher OR ($2.2$), still significant, and $M$ now has a much larger OR ($4.4$), this time significant. The interaction term is significant, with an OR of $0.2$.

I'm really surprised seeing the OR for the interaction term. My knowledge of the field pushes me to think something is wrong, as I would've expected the OR of the interaction term to be either above $1$ or not statistically significant.

My question is as follows: could this interaction term be the result of an imbalance between my two exposure groups? Does the fact that most people who took M were also people with exposure X matter? When I test $Y = M$ in only the $X = 1$ group, $M$ has a non-significant OR. When I test $Y = M$ in only the $X = 0$ group, $M$ has a large, significant OR. Should I even be testing for interaction when my exposure groups have such different sizes?

$\endgroup$
5
  • $\begingroup$ The conditions stated in your first paragraph contradict themselves. $0.05*29500 + 0.10*500 = 1525$ and not 1000. $\endgroup$
    – Peter Flom
    Nov 27, 2023 at 0:46
  • $\begingroup$ Could you provide the data as a 2x2x2 table. Currently you only show the marginal values. So that middle table could be split up into two levels $Y=0$ and $Y=1$ and look like for example $$\begin{array}{}&X=1&X=0\\M=1&10:90&9:40\\M=0&30:301&708:26040 \end{array}$$ or whatever other values you have. $\endgroup$ Nov 27, 2023 at 13:49
  • $\begingroup$ @SextusEmpiricus Done! :) $\endgroup$
    – awastus
    Nov 27, 2023 at 17:16
  • $\begingroup$ @awastus in the table of your second edit I would expect the sum of the cells to equal the values of the second table in your first edit. Why is this not the case? $\endgroup$ Nov 27, 2023 at 18:59
  • $\begingroup$ @SextusEmpiricus Ah, sorry. I should've added some info about this. This was due to missing data. Perhaps I should've included cases where X, M, and Y were all complete initially. In the latest table, it's complete data only, whereas in my edit before that, a table only showing X and M values didn't need to have complete Y values. $\endgroup$
    – awastus
    Nov 27, 2023 at 19:01

2 Answers 2

5
$\begingroup$

One problem is that you are relying far too much on significance tests and not showing us all the effect sizes. You say that in separate analysis by levels of X, the X=1 test was not significant while the X=0 test was. But the X=0 group is 59 times bigger than the X=1 group.

Your interaction says that the effect of M is smaller in X=1 than X=0. That is not inconsistent with the above, since p values are affected by sample size. But looking at overall main effects (as you do, in your second paragraph) is not sensible. Those main effects are the effect when the other variable is 0. The OR for M is 5 when X = 0. It is much lower when X = 1.

In a situation like this, where all the variables are dichotomous, a good way to see what is going on is to make a table with 4 rows with the conditions for X and M, and then the proportion with Y for each row.

EDIT in response to comment:

This is not what I meant, sorry for not being clearer. What I had in mind was something like this:

X     M     P(Y = 1)
0     0       
0     1          
1     0     
1     1

Since you are doing logistic regressions, I'm assuming that you are interested in the proportion of Y that are 1.

EDIT #2

Looking at your new table, when X = 1. M makes almost no difference. When X = 0, M makes a big difference. Similarly, when M = 1, X makes little difference, but when M = 0, X makes a huge difference.

That's an interaction, which is what you found.

$\endgroup$
2
  • $\begingroup$ My apologies Peter. In my attempts to simplify my question, I used approximations, which evidently did not hold up. I have made an edit to my original question, showing the actual data. I hope my question makes more sense now. Much appreciated. $\endgroup$
    – awastus
    Nov 27, 2023 at 4:30
  • $\begingroup$ No worries, it was my fault for not understanding. I added another table, as suggested by another commenter. I believe, although its format is a bit different from what you propose, it provides you the info you need. Do correct me if I'm mistaken, however. Thanks. $\endgroup$
    – awastus
    Nov 27, 2023 at 17:19
1
$\begingroup$

My knowledge of the field pushes me to think something is wrong, as I would've expected the OR of the interaction term to be either above 1 or not statistically significant.

If the interaction effect is absent, then this means that the for the log odds of $y$ for a particular category $x$ and $m$ the effects are additive.

$$\log \left( \frac{P(Y=1|x,m)} {P(Y=0|x,m)} \right) = a_0+a_1 x+a_2 m$$

If $x=1$ increases the odds by a factor $1.9$ and $m=1$ increases the odds by $1.7$, then in the absence of an interaction both together add up (or multiply of we consider odds instead of log odds) and increase the odds by $1.7 \times 1.9 = 3.23$.

Is that what you expect when you say that you expect the interaction effect to be absent? You say that you don't expect an interaction, which might mean that you consider the effects to be independent, but depending on how you measure or express the outcome (e.g. with a logistic link function or with something else) you get this interaction effect in any case.

The code below demonstrates that the significance of the interaction term can depend on the type of link function.

Example in R code:

n = 10000

### predictors 
x1 = rep(c(0,1),n)
x2 = rep(c(0,1),each = n)

### linear model for log probability 
p = exp(-0.6 + 0.2*x1 + 0.2*x2)

### simulate response
set.seed(1)
y = rbinom(2*n,1,p)

### this logistic model will have a significant interaction term
mod1 = glm(y~x1*x2, family = binomial(link = logit))
summary(mod1)

### this exponential model will have an insignificant interaction term
mod2 = glm(y~x1*x2, family = binomial(link = log))
summary(mod2)
$\endgroup$
3
  • $\begingroup$ Thanks for your comment. When I said I didn't expect this interaction, I meant that with what I know from the literature on the subject, everything seems to suggest that if there were to be an interaction between X and M, it should at least be positive. I would've therefore expected the interaction term to have an OR > 1. The fact that the interaction term is negative leads me to wonder if there was something I wasn't doing right. That's when I wondered if the fact that my groups are so imbalanced might be generating some biased result. I tested with log link instead of logit, similar results. $\endgroup$
    – awastus
    Nov 27, 2023 at 17:25
  • $\begingroup$ @awastus Maybe I am asking something obvious that you already thought of, but you did think about the fact that this interaction effect depends on the choice of the intercept? When you switch the labels then a positive interaction becomes a negative interaction. Also, in certain codes the order of the variables play a role (the intercept is chosen based on whatever label comes first). $\endgroup$ Nov 27, 2023 at 18:54
  • $\begingroup$ Thanks again for your comment. I see what you're saying, but no, this doesn't seem to be a labelling issue. I think that the interaction term will always suggest that people with both $X = 1$ and $M = 1$ will have less odds of $Y$ than people with only $X = 1$ or $M = 1$, given the data. I just wondered if there were some scenarios in which you shouldn't test for interactions, because the resulting interaction term could lead to deceiving results. I wondered if because of the particularities of my data (huge class imbalance for both $X$ and $M$), this is one of those scenarios. $\endgroup$
    – awastus
    Nov 27, 2023 at 19:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.