I'm studying an effect between 2 groups, and I am performing a mediation analysis to dive into the mechanism.
One question: After I calculated the mediation effect for each group, can I compare the 'coefficient' for each group and make some conclusion, or do I need to do some statistical tests to prove it?
Sorry if it sounds dumb but I'd like to make sure I'm not making some conclusion without following proper protocol.
Attaching a picture to show the conclusion I am trying to make. Thanks!
Nothing stopping you from eyeballing the coefficient(s), but as you intuit, that'd be a pretty low bar of evidence. There are a couple of comparative tests that you could (and probably should) conduct if establishing group differences in your indirect effects model is important:
"Moderated-mediation"/"Conditional process analysis":
This is probably the more popular, albeit IMO needlessly complicated/ambiguous, approach. If you are familiar with the idea of an interaction/moderation test, then apply that same notion to an indirect effects model like the one above. You could essentially calculate an interaction term between treatment (dummy or effect coded; though the choice would shape your interpretation) and $X$ for prediction of $Z$ and/or $Z$ for prediction of $Y$.
This approach is discussed extensively in Hayes (working on 3rd edition for 2021). While I like Hayes's conceptual coverage on the topic-- it's a pretty good introduction to various linear models that are common in social science--what I (and many of my colleagues) dislike is the software implementation, which relies on Hayes's PROCESS macro for SPSS. To make a long rant short: the issue with PROCESS is that it allows users to specify a bazillion different models corresponding to different patterns of moderation and mediation, when there is fairly clear evidence that many among its user-based don't actually understand how to choose a model with a good rationale, or to interpret the estimates it yields (interaction terms are rarely intuitive to beginners), which is (if you looked around) how you end up with a mediation tag on CV that is dominated by questions about PROCESS models.
In your case, however, the choice of model would be relatively limited between what path(s) ($X$ --> $Z$ and/or $Z$ --> $Y$ holding $X$ constant) you wanted to indulge as possibly different between treatment groups. So I would consider whether you are comfortable specifying/interpreting interaction terms in regression, and if you feel good about that, then this could be a feasible way to approach the comparison.
Your alternative to this approach would be...
Structural Equation Modelling (SEM):
Before you fret: this would be a very, very simple/straightforward usage of SEM (which otherwise can get very complicated). This approach to comparing paths in indirect effect models is less common, but IMO, it's way more streamlined/intuitive when dealing with a grouping variable (like your case of treatment vs. control).
SEM is neat for many reasons (see here for an older thread that provides a good overview), but one relevant here is that the user can dictate values they think ought to be the case in their model (e.g., the value of a particular slope), program that value in, and evaluate how reasonable it is (see the 4th "Why use SEM in the first place?" reason that I outline in that thread). In a (very brief) nutshell, one of the valuable pieces of information SEM provides the user is a metric of how closely the statistical properties of their variables (namely, variances, covariances, and sometimes means) that are implied by their specified model track the actual (or "observered") statistical properties of those models--this is what is meant by "model fit" (better fitting models have a smaller gap between model-implied and observed values). Researchers can therefore compare model fit between a model in which they program in their dictated values (called a "constraint") and one in which they don't and it's "freely estimated".
In your case, you would be in comparing a model in which your indirect effect pathways ($X$ --> $Z$ and/or $Z$ --> $Y$ holding $X$ constant) were held constant between treatment and control. Learning SEM (in general, or for this specific purpose) will take a bit of investment (I strongly recommend Beaujean, 2014, for a quick primer), but thankfully making these kinds of constraints for a model like yours would not be horribly complex. Here is what it could look like (using syntax from the lavaan package, which is a go-to open source option):
free.model <- '
Y ~ c(cp1, cp2)*X
Z ~ c(a1, a2)*X
Y ~ c(b1, b2)*Z'
constrained.model <- '
Y ~ c(cp1, cp2)*X
Z ~ c(a, a)*X
Y ~ c(b, b)*Z'
If you fed each of these models to a model-fitting function for lavaan, and chose your treatment variable as a grouping variable, the first model would estimate two unique slopes (one for each group) for each pathway in your model; the second model would estimate a shared slope for each of your pathways of potential interest, "forcing" (or "constraining" them to be equal). The way we tell the software which pathways to estimate freely vs. constraint to equality between groups is via the "labels" you see. Unique labels for a given pathway (e.g., a1 and a2 for $Z$ ~ $X$; b1 and b2 for $Y$ ~ $Z$ holding $X$ constant), as in the first "free.model", ensures that each group will have their own unique slope value (which could be superficially similar or different from one another). Identical labels for a given pathway (e.g., a and a for $Z$ ~ $X$; b and b for $Y$ ~ $Z$ holding $X$ constant), as in the second "constrained.model", forces these pathways to take on a shared value. Once both models were fit, you could then compare the model fit values of each model, a process that is little more than a glorified likelihood ratio test, which can be carried out with syntax as simple as something like:
anova(model1, model2)
tl;dr:
Your options are conducting "moderated-mediation" tests via something like PROCESS, or comparing a couple of models in SEM (one where pathways are freely estimated for each group vs. one where they are constrained to equality between groups). Though each approach will answer your question, they approach the question somewhat different in terms of how the statistical values capture the similarity or difference of your indirect effects model.
The "moderated-mediation" essentially asks, "How much does the slope of my pathway(s) change as a function of the level of my treatment variable?" (which it captures in an interaction term). The SEM approach, meanwhile, essentially asks "How much does the quality of my indirect effects model degrade if I assume the model estimates are the same for each treatment group?" (which it then captures in the difference in model fit between your two competing models).
References
Beaujean, A. A. (2014) Latent Variable Modeling Using R: A Step-by-Step Guide. Routledge: New York, New York.
Hayes, A. (2021). Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach. Guilford Press: New York, New York.
