Which is the measurement invariance level necessary for a multiple regression analysis?

Question

I am interested in running a multiple regression model to test the association between a predictor variable x and an outcome variable y and control by some confounders variables (e. g. gender, age). Considering that metric invariance (loadings constrained to be equal across groups) across confounder variables groups (gender, age) is supported for the predictor variable and for the outcome variable, is it ok to run multiple regression models (examples : binomial negative , logistic regression) or is it necessary to have also scalar invariance ( loadings and intercepts constrained) ?

Answer 1

It depends on what the substantive question you are trying to answer is, but in general scalar invariance is required for group comparisons. Let's say you have the following regression model:

$y = α + β_1x + β_2gender + β_3(gender×x) + ε$

where $y$ is the outcome variable, $x$ is the predictor variable, $gender$ is a covariate, $α$ is an intercept and $β_1, β_2$ and $β_3$ are the coefficients of the main effect of $x$, the main effect of gender and the moderating effect of gender, respectively. $ε$ is a residual.

If metric invariance holds for $y$ and $x$, $E[β_3] = 0$ if gender has no moderating effect on $y$. Therefore, if $β_3 ≠ 0$, gender moderates the effect of $x$ on $y$ for substantive reasons (i.e. it's not a scale artifact).

If scalar invariance doesn't hold for $x$ and/or $y$, $E[β_2] ≠ 0$ even if in fact $x$ fully explains the relationship between $gender$ and $y$. The biased main effect of gender arises from the fact that with scalar non-invariance, individuals from different groups don't have the same expected observed $x$ or $y$ values conditional on having the same values on the underlying, latent attribute(s). Therefore, with scalar non-invariance, it is not possible to say whether a significant $β_2$ coefficient means that $gender$ has a main effect on $y$ or whether the apparent effect is just a scale artifact.

There are further complications when regression equations in different groups are compared. Even if strict measurement invariance holds, regression equations are not expected to be the same if the groups differ in their means or variances with respect to the latent attribute underlying the independent variable $x$. See Millsap (2007) for an explication of why measurement invariance is generally inconsistent with predictive invariance.