Suppose there is one independent variable (X) and a moderator (M), and both are continuous. The objective is to study the interaction effect of XM on Y. As X or M may have high multicollinearity with XM, therefore following is performed. X is centered, M is centered but however for XM, z-score is taken and then it is squared. Is it possible to do that in multiple regression? What reference can be quoted here..
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
As often, writing down the different models is clearer than talking about them. If your model is
$$\operatorname{E} Y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_{12} \left(x_1x_2\right)$$
& you decide to centre & scale some predictors using constants $a,b,c,d$ (which can be means, standard deviations of the predictors, or whatever), the new model is
$$\operatorname{E} Y = \beta_0^* + \beta_1^* (x_1 -a) + \beta_2^* (x_2-b) + \beta_{12}^* \frac{x_1x_2 - c}{d}\\ =\beta_0^* -a\beta_1^*-b\beta_2^*-\frac{c\beta_{12}^*}{d}+\beta_1^*x_1 + \beta_2^* (x_2-b) + \frac{\beta_{12}^*}{d} (x_1x_2)$$
But this is just a reëxpression of the old model, with:
$$\beta_0=\beta_0^* -a\beta_1^*-b\beta_2^*-\frac{c\beta_{12}^*}{d}\\ \beta_1=\beta_1^*\\ \beta_2=\beta_2^*\\ \beta_{12}=\frac{\beta_{12}^*}{d}\\ $$
The predictions & prediction intervals are exactly the same; you've just changed the units the predictors & coefficients are measured in.
On the other hand
$$\operatorname{E} Y = \beta_0' + \beta_1' (x_1 -a) + \beta_2' (x_2-b) + \beta_{12}' \left(\frac{x_1x_2 - c}{d}\right)^2$$
is a brand new model with an $(x_1x_2)^2$ term where before you had only $x_1x_2$. It will make different predictions; moreover, as @Peter pointed out, those predictions will depend on the units the predictors are measured in—e.g. Celsius vs Fahrenheit—if you don't include the lower-level terms as well, so you'd better be sure you really want that. The form of the interaction is unusual: when $x_2=0$, the response is linearly related to $x_1$; as $x_2$ increases in absolute magnitude the relation becomes a tighter parabola, with a maximum or minimum depending on the sign of $\beta'_{12}$—it's hard to imagine this is what you had in mind. It's sometimes said such models violate the marginality principle: see Venables (1998), "Exegeses on linear models", S-Plus Users' Conference, Washington DC.
Collinearity is a red herring here—it's not a problem when it's a matter of how the predictors are defined (sometimes called structural collinearity) because you're never going to have to predict responses for predictor patterns that aren't exhibiting exactly the same collinearity.
NB You hear different views about centring & scaling predictors, discussion of which often drags in collinearity. It's important to realize that these concern ease of interpretation only (or occasionally accuracy of calculation—not an issue with algorithms used by modern software, as @Frank said).
answered Feb 16, 2014 at 17:49
-
$\begingroup$ Thanks for elaborate answer sir. But just to confirm again, i am not squaring x1x2 like you showed above, rather i am squaring the z-score of x1x2. Along with that in the regression model, i include x and x2 also along with certain other control variables. Does it change your viewpoint now? $\endgroup$ Commented Feb 17, 2014 at 5:23
-
$\begingroup$ Thanks for pointing that out; I've altered the formula of the last model to reflect it. But it makes no difference to my viewpoint - which isn't a 'viewpoint', just the maths we learnt at school. Draw some graphs if the behaviour of the model isn't apparent, considering the consequences of what you're holding fixed & what you're allowing to vary. For instance, when $x_2=0$, the response varies linearly with $x_1$ - is that a reasonable restriction to impose before estimating any coefficient? $\endgroup$– Scortchi ♦Commented Feb 17, 2014 at 10:01
$\begingroup$
$\endgroup$
10
Centering before multiplying is a good idea but there is no need to take the z-score, much less to square it. Note that XM should be on the centered variables, so, really, more like $X_cM_c$ where the subscript indicates "centered".
answered Feb 16, 2014 at 15:38
-
1$\begingroup$ Centering only helps in a very narrow sense, I think. It will not effect the estimates of the interaction parameters; it will only change the "main effects" which anyhow are not so interpretable in the presence of interaction. $\endgroup$ Commented Feb 16, 2014 at 15:44
-
1$\begingroup$ Squaring the interaction may solve the collinearity problem but it also changes its identity into the interaction of $X^2$ and $M^2$. For that reason, I'd recommend not to call it "adjusting for interaction," simply because the model really is not doing that. $\endgroup$ Commented Feb 16, 2014 at 16:04
-
1$\begingroup$ Transforming XM with a z score will not substantively change the model. If you want to look at $X^2M^2$ then you can do so (although you should then probably include X and M and XM) $\endgroup$ Commented Feb 16, 2014 at 16:26
-
2$\begingroup$ Yes I disagree with the practice. If using modern software (e.g., R) centering doesn't help, and it makes interpretation more difficult. The Q-R decomposition is insensitive to extreme collinearity, and collinearity does not affect predicted values nor does it hurt hypothesis testing, at least for interesting hypotheses. $\endgroup$ Commented Feb 16, 2014 at 16:55
-
1$\begingroup$ @user39496: To look at the $XM$ interaction you need to look at it, not at something else. Why do you think collinearity between $X$ & $XM$ or between $M$ & $XM$ is any sort of problem? Will you be needing to predict the response when $X=2$ & $M=3$ & $MX=10$? $\endgroup$– Scortchi ♦Commented Feb 16, 2014 at 17:00