1
$\begingroup$

I have encountered an interesting phenomenon while working on some data derived from a(n admittedly badly constructed) questionnaire by a colleague, and I have no idea what to make of it mathematically speaking, even though I am familiar with multiple regression and suppression.

The phenomenon is as follows:

There are three independent variables (call them I1, I2 and I3), and a single dependent variable (D), all calculated by summing/averaging over subscales in a psychometric instrument). The aim is to fit a multiple regression model and test whether I1, I2 or I3 are significant predictors of D. However, the data produces the following results:

When only I1 is used to predict D, the result is very not significant (p = 0.613)

When only I2 is used to predict D, the result is, again, not significant. (p = 0.09)

When only I3 is used to predict D, the result is also not significant. (p = 0.999)

However, when I2 and I3 are included together in the model, both coefficients are significant (p < 0.01 for both of them). This is not affected by also including I1 as a third IV. In this case, I2 and I3 retain their significance, but I1 will not become significant.

This looked suspiciously like Suppression/Collinearity to me, so I checked correlations between the independent variables. All 3 IVs are significantly correlated with each other (0.65, 0.79 and 0.83) but none of them are significantly correlated with D by themselves (-.042, -0.12, -0.022, p > 0.05 in all cases). I then checked VIF and Tolerance values. They seem to signal that something is wrong (highest VIF is around 4.5, lowest tolerance is 0.2) but they don't seem excessive.

What I find strange (and unlike other Suppression cases I have seen before, where only one IV was made significant by the addition of another) is that BOTH I2 and I3 become significant when they are both added at the same time, but none of them are made significant either by the addition of I1 in the 3-IV model, or in individual 2-IV models (I1-I2 and I1-I3).

Mathematically, what may be the cause of this behavior? I would be glad to provide more information if necessary. I also have not been able to find a thread with exactly the same issue.

$\endgroup$

1 Answer 1

2
$\begingroup$

We'll ignore I1 in the model since there is no evidence it affects the outcome in any way. Consider the following system of structural equations governing this system: \begin{align} I_2 &= u_2 \\ I_3 &= \beta_{32}I_2 + u_3\\ Y &= \beta_{Y2}I_2 + \beta_{Y3}I_2 + u_Y \end{align} This corresponds to the classic confounder/mediator graph, where $I_2$ is a confounder for the $I_3 \rightarrow Y$ relationship, and $I_3$ is a mediator between $I_2$ and $Y$.

enter image description here

Let $I_2$, $I_3$, and $Y$ have a standard deviation of $1$. If $\beta_{32} \approx 1$ and $\beta_{Y2}=-\beta_{Y3}\ne 0$, the phenomenon you observed will occur. This is because the marginal association between $I_2$ and $Y$ is $\beta_{Y2} + \beta_{32}\beta_{Y3}$ and the marginal association between $I_3$ and $Y$ is $\beta_{Y3} + \beta_{32}\beta_{Y2}$. Under the conditons I mentioned, the marginal association between $I_2$ and $Y$ is close to $0$ and the marginal association between $I_3$ and $Y$ is also close to $0$ (i.e., yielding nonsignificant p-values in the single regressions), but $\beta_{Y2}$ and $\beta_{Y3}$ are not $0$, yielding significant p-values in the multiple regression of $Y$ on $I_2$ and $I_3$. There are other conditions in which this will occur, and you can use path analysis and a little algebra to discover them.

$\endgroup$
2
  • $\begingroup$ Thanks for the detailed response! Could you clarify some terminology for the structural equations? What does "u2" stand for and how do the subscripts for the regression coefficients (32, Y2, Y3) come about? Also, could you perhaps show me a visual representation of the structural model in question? I am not extremely sophisticated mathematically and it would help a lot. Thanks! $\endgroup$ Apr 18, 2021 at 14:43
  • $\begingroup$ The "u"s represent error terms, i.e., random variation unrelated to the other variables, like the error in a regression. The subscripts are arbitrary but I used the convention where the first subscript is the outcome and the second subscript is the predictor, so $\beta_{Y2}$ is the coefficient from $I_2$ to $Y$. I added an image of the DAG. $\endgroup$
    – Noah
    Apr 18, 2021 at 16:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.