1
$\begingroup$

This answer states that the analysis done through the R mediation package:

isn't designed to prove that there is no direct effect, but rather show that there is one. Your insignificant estimate for the ADE reflects a lack of evidence, not contrary evidence per se.

So how can I prove that there is no direct effect in a model?

$\endgroup$
0

2 Answers 2

1
$\begingroup$

You are looking for a test for equivalence (general to statistical inference, and not restricted to mediation analysis). A simple approach is to use two one-sided tests for equivalence. Uniformly most powerful tests for equivalence are a more sophisticated approach.

Generally, one chooses a minimum magnitude of difference $\delta$ that you care about (i.e. a minimum effect size): any difference $\delta$ or smaller indicates quantities which are equivalent for your purposes. The null hypothesis for a test for equivalence is $\text{H}_{0}\text{: } |\theta| \ge \delta$. (In plain language "the magnitude of the difference between quantities is at least as big as $\delta$.") And the alternative hypothesis for a test for equivalence test $\text{H}_{\text{A}}\text{: } |\theta| < \delta$. (In plain language "the magnitude of the difference between quantities is smaller than $\delta$.") If one rejects $\text{H}_0$, then you have found evidence that $\theta < \delta$ and $\theta > -\delta$. I.e. you have found evidence that $-\delta < \theta < \delta$.

One can express this equivalence interval asymmetrically (often used for asymmetrically-scaled measures, such as odds ratios, etc.).

Finally, you can also combine inference from a test for difference with a test for equivalence (and thereby guard against confirmation bias). See the 2×2 table in my answer for the logic of combined inference in so-called relevance tests.

Some useful references
Hauck, W. W., & Anderson, S. (1984). A new statistical procedure for testing equivalence in two-group comparative bioavailability trials. Journal of Pharmacokinetics and Pharmacodynamics, 12(1), 83–91.

Schuirmann, D. A. (1987). A Comparison of the Two One-Sided Tests Procedure and the Power Approach for Assessing the Equivalence of Average Bioavailability. Journal of Pharmacokinetics and Biopharmaceutics, 15(6), 657–680.

Wellek, S. (2010). Testing Statistical Hypotheses of Equivalence and Noninferiority (Second Edition). Chapman and Hall/CRC Press.

$\endgroup$
0
$\begingroup$

To add to @Alexis's answer, the issue is not with the R package but with null hypothesis testing. When using a null hypothesis test (where we want to reject the null hypothesis and will do so when p is less than some value), then it is impossible to ever accept the null hypothesis - we can only reject it. Specific equivalence tests exist to address this limitation since testing for equivalence is a different statistical question than testing for different (the primary use of null hypothesis tests).

One way around this is, of course, the use of equivalence testing, but a more universal approach is also just a Bayesian methodology. While Bayes can be intimidating to get into, the brms package is very familiar to anyone running regression in R. There are many examples and tutorials of using either brms or rstanarm for mediation purposes (e.g., here), and the bayestestR package gives a very similar layout and function to the mediation package. If you use flat priors (which are the default for brms), then you actually get the same regression coefficient results as you would with maximum likelihood estimation from standard regression.

With the bayestestR package, there is a function that will allow you to request the ROPE (region of practical equivalence) for each coefficient. The ROPE provides an interval of what we would consider practically equivalent to 0 (e.g., coefficients between -0.1 and 0.1 are practically zero). To test whether the direct effect is non-existent, you're hypothesizing that its coefficient is within this ROPE. What you get from a Bayesian mediation that you don't get from frequentist methods is that your summary is probabilistic. In other words, you're not answering with a binary "yes" or "no" as to whether your direct effect is zero; instead, you're able to say the probability that the direct effect is practically zero. You could also consider computing the Bayes factor for the null hypothesis (no effect) against the alternative hypothesis (some effect). The Bayes factor is the ratio of odds favoring one or another hypothesis.

The other thing you get from a Bayesian approach is the ability to summarize evidence for and against either the null or alternative hypothesis right out of the box. With more and more helpful packages like brms, it's becoming increasingly easy to default to Bayesian methods for these kinds of flexible methods. While flat priors (technically it needs to be proper uniform priors) get you the same results as maximum likelihood, using flat priors with Bayesian methods is a bit like never driving more than 30mph in a sports car. Still, specifying priors is a vital part of Bayesian methods and gets easier once you've got some Bayes under your belt

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.