10
$\begingroup$

In a classic mediation model, we have paths shown in the diagram below

,

in which the first step of testing the mediating effect of M between X and Y is that X is significantly correlated with Y (as shown in panel A in the figure).

However, I bumped into a situation where Path a and Path b are strongly significant, but not Path C. Compared to Path c, Path c' is not significant, but the coefficient is decreased.

In this case, is it still valuable to talk about the relationship among X, Y, and M?? If so, what is the best way to address this relationship in a paper? Can we claim that X has an indirect effect but not a direct effect on Y??

I am testing the same path model with three samples, $n_1 = 124, n_2 = 49, n_3 = 166$.

$\endgroup$
  • 1
    $\begingroup$ Please add the sign of the effects and magnitude. If you've got a low N and just don't have a significant effect of c but it's nevertheless a moderate number it means something different than if c is very closer to 0. $\endgroup$ – John Dec 8 '15 at 22:42
28
+50
$\begingroup$

Your approach to testing mediation appears to conform to the "causal steps approach" described in the classic methods paper by Baron & Kenny (1986). This approach to mediation entails the following steps:

  1. Test whether X and Y are significantly associated (the c path); if they are not, stop the analysis; if they are...
  2. Test whether X and M are significantly associated (the a path); if they are not, stop the analysis; if they are...
  3. Test whether M and Y are significantly associated after controlling for X (the b path); if they are not, stop the analysis; if they are...
  4. Compare the direct of effect of X (the c' path--predicting Y from X after controlling for M) to the total effect of X (the c path from Step 1). If c' is closer to zero than c, and non-significant, the research concludes that M completely mediates the association between X and Y. But if c' is still significant, the researcher concludes that M is only a "partial" mediator of X's influence on Y.

I emphasize the difference between direct (c') and total effects (c) because though you wrote...

Can we claim that X has an indirect effect but not a direct effect on Y??

I think what you are actually concerned about is the legitimacy of claiming that X has an indirect, but not a total effect on Y.

The Short Answer

Yes, it is legitimate to conclude that M mediates the association between X and Y even if the total effect (c) is not significant. The causal steps approach, though historically popular, has been widely replaced by methods of testing for mediation that are more statistically powerful, make fewer assumptions of the data, and are more logically coherent. Hayes (2013) has a wonderfully accessible and thorough explanation of the many limitations of the causal steps approach in his book.

Check out other more rigorous approaches, including the bootstrapping (MacKinnon et al., 2004) and Monte Carlo (Preacher & Selig, 2012) methods. Both methods estimate a confidence interval of the indirect effect itself (the ab path)--how they do so differs between methods--and then you examine the confidence interval to see whether 0 is a plausible value. They are both pretty easy to implement in your own research, regardless of which statistical analysis software you use.

The Longer Answer

Yes, it is legitimate to conclude that M mediates the association between X and Y even if the total effect (c) is not significant. In fact, there is a relatively large consensus among statisticians that the total effect (c) should not be used as a 'gatekeeper' for tests of mediation (e.g., Hayes, 2009; Shrout & Bolger, 2002) for a few reasons:

  1. The causal steps approach attempts to statistically evaluate the presence of mediation without ever actually directly evaluating the indirect effect (the ab path, or c-c' if you prefer). This seems illogical, especially given that there are numerous easy ways to estimate/test the indirect effect directly.
  2. The causal steps approach is contingent on multiple significance tests. Sometimes significance tests work as they should, but they can be derailed when assumptions of inferential tests are not met, and/or when inferential tests are underpowered (I think this is what John was getting at in his comment on your question). Thus, mediation could be really happening in a given model, but the total effect (c) could be non-significant simply because the sample size is small, or assumptions for the test of the total effect have not been met. And because the causal steps approach is contingent on the outcome of two other significance tests, it makes the causal steps approach one of the least powerful tests of mediation (Preacher & Selig, 2008).
  3. The total effect (c) is understood as the sum of the direct effect (c') and all indirect effects (ab(1) , ab(2)...). Pretend the influence of X on Y is fully mediated (i.e., c' is 0) by two variables, M1 and M2. But further pretend that the indirect effect of X on Y through M1 is positive, whereas the indirect effect through M2 is negative, and the two indirect effects are comparable in magnitude. Summing these two indirect effects would give you a total effect (c) of zero, and yet, if you adopted the causal steps approach, you would not only miss one "real" mediation, but two.

Alternatives that I would recommend to the causal steps approach to testing mediation include the bootstrapping (MacKinnon et al., 2004) and Monte Carlo (Preacher & Selig, 2012) methods. The Bootstrapping method involves taking a superficially large number of random samples with replacement (e.g., 5000) of the same sample size from your own data, estimating the indirect effect (the ab path) in each sample, ordering those estimates from lowest to highest, and then define a confidence interval for the bootstrapped indirect effect as within some range of percentiles (e.g., 2.5th and 97.5th for a 95% confidence interval). Bootstrapping macros for indirect effects are available for statistical analysis software like SPSS and SAS, packages are available for R, and other programs (e.g., Mplus) have bootstrapping capabilities already built-in.

The Monte Carlo method is a nice alternative when you don't have the original data, or in cases when bootstrapping isn't possible. All you need are the parameter estimates for the a and b paths, each path's variance, and the covariance between the two paths (often, but not always 0). With these statistical values, you can then simulate a superficially large distribution (e.g., 20,000) of ab values, and like the bootstrapping approach, order them from lowest to highest and define a confidence interval. Though you could program your own Monte Carlo mediation calculator, Kris Preacher has a nice one that is freely available to use on his website (see Preacher & Selig, 2012, for accompanying paper)

For both approaches, you would examine the confidence interval to see if it contains a value of 0; if not, you could conclude that you have a significance indirect effect.

References

Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173-1182.

Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York, NY: Guilford.

Hayes, A. F. (2009). Beyond Baron and Kenny: Statistical mediation analysis in the new millennium. Communication Monographs, 76 408-420.

MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99-128.

Preacher, K. J., & Selig, J. P. (2012). Advantages of Monte Carlo confidence intervals for indirect effects. Communication Methods and Measures, 6, 77-98.

Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychological Methods, 7, 422-445.

$\endgroup$
  • $\begingroup$ Thanks! It was very helpful! I had several models that the upper bond of CI is so close to 0, for example, the CI shown in my result is [-.1549, .0031]. Does it mean I still need to reject the null hypothesis? I used PROCESS and bootstrapping for my analysis. $\endgroup$ – fishbean Dec 11 '15 at 8:23
  • $\begingroup$ You would fail to reject the null; 0 falls between -.1549 and .0031, so you cannot conclude that the ab path is greater than zero. $\endgroup$ – jsakaluk Dec 11 '15 at 13:18
  • 2
    $\begingroup$ +6, this is the kind of answer we would like more of on CV. I hope having gotten only 1 upvote won't discourage you from providing more in the future. Perhaps a bounty can help get this thread the attention it deserves. $\endgroup$ – gung Jun 11 '16 at 16:42
  • $\begingroup$ I'm sorry this didn't bring more upvotes, @jsakaluk. This is a great answer; I hope you'll continue to provide them. $\endgroup$ – gung Jun 18 '16 at 18:07
  • $\begingroup$ No worries @gung. I appreciate the support. I had taken a step back from CV after investing a lot of time in a previous question that ended up being highly voted, only to have it very quickly made CW (bit of a bummer), so this experience was a nice encourager. $\endgroup$ – jsakaluk Jun 18 '16 at 18:10
0
$\begingroup$

Okay, I think I might have found a good answer. I took a look at David Kenny's webinar, which introduces this case as inconsistent mediation. The reason why path c is not significantly different from 0 is that the product of a and b has a sign different from that of c'. In an example Kenny gives, stress leads to an decrease in mood (c' is negative); whereas exercise as a mediator between stress and mood is positively correlated to both (ab is positive). Since c = c' + ab, when the absolute values of c' and ab are close, c could be close to 0.

Kenny notes in the webinar that contemporary view considers testing of c and c' not quite essential; the mediation effect is mainly displayed through ab.

$\endgroup$
  • $\begingroup$ For the sake of posterity, could you use a single set of variable names in your diagram and text? Currently you've got "IV" and "X" and "Stress" all denoting the same thing, and so on. $\endgroup$ – rolando2 Dec 8 '15 at 11:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.