2
$\begingroup$

I am conducting a path analysis that include estimation of indirect effects (i.e. mediation). For each indirect effect specified in the model (a1*b1), I have both a p-value and a 95% boostrapped CI.

I see that a similar question has been asked about t-tests but I am not sure what the take away is for my situation (p points to non-sig, CI suggests significant effect, not t-test).

standardized beta=.08, p=0.12, ci=0.003-0.16 (10000 bootstrapped CI)

(1) Is it possible to choose p-value vs. the CI? If so, what is the rationale? (citations/references would be appreciated)

(2) This occurs across several models I am running. Does this speak to a broader problem that I have failed to consider? All analyses completed in Lavaan package in R.

$\endgroup$

1 Answer 1

1
$\begingroup$

Reading the lavaan tutorial, by default standard errors are Delta standard errors page 34. The bootstrap CI of the regression coefficient, on the other hand, is an empirical estimate of the variance.

The disagreement between your significance tests is due to the differences in the methods used - the Delta method is semi-parametric, and the empirical bootstrap is non-parametric.

Which you choose is really up to you (and will relate to the assumptions you have about your data and model and the norms in the field you are working in - can you assume, for example that your SE is unbiased? do you have un-modelled dependencies within your data like hierarchical structures?). If you choose to use the bootstrap method, lavaan can handle this - see page 32 of the tutorial.

The more important question may be to ask, are you really interested in such marginal findings? The fact you need to report the lower bound of your CI to 3 decimal places shows how close your CI is to including zero.

$\endgroup$
2
  • $\begingroup$ This is an old post, but on the off-chance that you still read this: based on your explanation, I would have expected p.values and CIs to agree when test = "bootstrap" is set. However, I encounter the same issue as the OP with that option turned on and can't fint documentation re how bootstrapped p.values are calculate by lavaan. Any ideas where the remaining discrepancy comes from? $\endgroup$ Jan 1, 2021 at 17:09
  • 1
    $\begingroup$ @LukasWallrich - the bootstrap estimation of the parameters in the model has a stochastic element – the resampling with replacement (by default, naive bootstrap - page 3 of the manual). Empirical CIs and pvalues may not agree because the CI is related to the variance of the bootstrapped param est, whilst the pvalue is related to the resulting average of the bootstrapped hypothesis tests (see distributional assumptions). Have you reviewed the distribution of your data? See also “test” in the section for lavOptions $\endgroup$
    – danCloney
    Jan 5, 2021 at 5:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.