1
$\begingroup$

I'm currently doing a multiple linear regression analysis on Spirituality and Forgiveness to see how it affects Life Satisfaction.

If I put the IVs individually into linear regression they both show up as significant, with spirituality being <.001 and forgiveness being .004. However, if I put both of them into the model spirituality stays as 0.001 while forgiveness goes to .543. I'm not really sure how to interpret these results.

Additionally I am doing a moderation analysis to see if spirituality moderates forgiveness shows insignificant moderation.

I checked for mediation, with spirituality being the mediator for forgiveness and life satisfaction. The results are as follows:

  • Direct effect of forgiveness on life satisfaction is shown to be insignificant

  • Indirect effect where spirituality is the mediator is shown to be highly significant at <.001.

Am I correct in assuming that the interpretation for this is full mediation i.e forgiveness only has a significant effect on life satisfaction if mediated by spirituality; otherwise there is no significant effect whatsoever as well as spirituality on its own being a predictor for life satisfaction as well?

$\endgroup$

2 Answers 2

1
$\begingroup$

I think the way you have applied these models (5 as far as I can count) sounds a little too exploratory to really lend too much weight to interpreting the $p$ values anymore. It doesn't really make sense to fit all of these models unless there is a strong theoretical basis for them before fitting. I anyway doubt the relationships can be easily justified as simultaneously independent, correlated, mediated, and moderated all at the same time. If you have a strong theoretical justification for one or more models, they should be made explicit so as to not motivate a fishing expedition with $p$ value.

This is especially critical for mediation analysis (see The Mediation Myth here specifically for some of the causal elements you need, which include a temporal ordering of your effects here). In your case, you have forgiveness -> spirituality -> life satisfaction. How do we know these variables follow this order? Couldn't life satisfaction predict forgiveness? Or spirituality predict forgiveness? I just see reverse causality all over this model. This is not an arbitrary point, and is necessary to have interpretable mediation models.

On to your actual question, the statistical significance can be affected by any number of things. Adding variables in can cause some to suppress the effect of others. Omitted variable bias can also be a culprit. Every time you fit a model, you are splitting up the estimable variance in your outcome by more potential pieces. Naturally, this can change how the $p$ values look. More important is the magnitude of the parameters, the uncertainty around those parameters, and their theoretical implications. The $p$ values can be useful, but often they're just not (see discussion here for example about practical vs statistical significance).

If there were defensible reasons for your multiple models (again, made before fitting them), then you could consider model comparison criteria like AIC/BIC, or LRT tests if you want a $p$-value based approach.

$\endgroup$
4
  • $\begingroup$ "It doesn't really make sense to fit all of these models unless there is a strong theoretical basis for them before fitting." The OP has only fit two or three models (with both IV included or each IV included one at a time. Unless I'm missing something, I don't think what the OP has done is anything unusual in typical regression analysis, when attempting to find the most parsimonious model. It's not like the OP has fit thousands of models here. That being said, I do generally agree that theory should typically drive model development, but I think the theory applied is fine. $\endgroup$ Commented Oct 31 at 14:08
  • $\begingroup$ @StatsStudent I don't necessarily agree with your statements here (see my edits), as they seem to have 5 very different models, one of which requires strong causal assumptions to fit. $\endgroup$ Commented Oct 31 at 14:24
  • $\begingroup$ Thanks for the response, can you elaborate on what "strong theoretical justification" means? Since we do have multiple studies and a theoretical framework that connects forgiveness to life satisfaction (or similar terms to life satisfaction). In regards to my study, I'm most likely going to drop moderation all-together as it showed insignificant results and will most likely either go for Spirituality as an IV or as a mediator (possibly both if that's okay?). Since there are previous studies that support these as well. $\endgroup$
    – user442416
    Commented Oct 31 at 15:54
  • 1
    $\begingroup$ You shouldnt just drop a model after its non sig. The point is to have a priori beliefs about models, testing them, then reporting them...rather than simply fitting things for the sake of fitting them. For mediation, you have to argue one variable precedes the other in time. I dont see that being defensible here. $\endgroup$ Commented Oct 31 at 16:25
0
$\begingroup$

Given that each IV variable produces a statistically significant result when included individually, but when both are included spirituality remains significant while forgiveness becomes non-significant indicates to me that there is multicollinearity present in your data and that both spirituality and forgiveness are correlated measures. Therefore, both spirituality and forgiveness share some of the variance in predicting life satisfaction, but spirituality captures most of the the shared variance. So, once spirituality is accounted for, adding forgiveness into your model adds little additional predictive power, and further reduction in explained variance.

With the direct effect of forgiveness on life satisfaction being nonsignificant, but the indirect effect (through spirituality) being highly significant, you’re correct in interpreting this as full mediation. This implies that forgiveness impacts life satisfaction only through its relationship with spirituality; without spirituality as a mediator, forgiveness does not have a significant effect on life satisfaction. Additionally, since spirituality remains a significant predictor in both the regression model and the mediation pathway, it has a unique, direct association with life satisfaction beyond mediating forgiveness.

It seems forgiveness influences life satisfaction through spirituality (full mediation), while spirituality independently predicts life satisfaction. This means spirituality is a central factor in your analysis, while forgiveness is impactful primarily through its relationship with spirituality.

$\endgroup$
1
  • $\begingroup$ Thanks! that definitely clears some stuff up $\endgroup$
    – user442416
    Commented Oct 31 at 16:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.