Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Suppose there is one independent variable (X) and a moderator (M), and both are continuous. The objective is to study the interaction effect of XM on Y. As X or M may have high multicollinearity with XM, therefore following is performed. X is centered, M is centered but however for XM, z-score is taken and then it is squared. Is it possible to do that in multiple regression? What reference can be quoted here..

share|improve this question

As often, writing down the different models is clearer than talking about them. If your model is

$$\operatorname{E} Y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_{12} \left(x_1x_2\right)$$

& you decide to centre & scale some predictors using constants $a,b,c,d$ (which can be means, standard deviations of the predictors, or whatever), the new model is

$$\operatorname{E} Y = \beta_0^* + \beta_1^* (x_1 -a) + \beta_2^* (x_2-b) + \beta_{12}^* \frac{x_1x_2 - c}{d}\\ =\beta_0^* -a\beta_1^*-b\beta_2^*-\frac{c\beta_{12}^*}{d}+\beta_1^*x_1 + \beta_2^* (x_2-b) + \frac{\beta_{12}^*}{d} (x_1x_2)$$

But this is just a reëxpression of the old model, with:

$$\beta_0=\beta_0^* -a\beta_1^*-b\beta_2^*-\frac{c\beta_{12}^*}{d}\\ \beta_1=\beta_1^*\\ \beta_2=\beta_2^*\\ \beta_{12}=\frac{\beta_{12}^*}{d}\\ $$

The predictions & prediction intervals are exactly the same; you've just changed the units the predictors & coefficients are measured in.

On the other hand

$$\operatorname{E} Y = \beta_0' + \beta_1' (x_1 -a) + \beta_2' (x_2-b) + \beta_{12}' \left(\frac{x_1x_2 - c}{d}\right)^2$$

is a brand new model with an $(x_1x_2)^2$ term where before you had only $x_1x_2$. It will make different predictions; moreover, as @Peter pointed out, those predictions will depend on the units the predictors are measured in—e.g. Celsius vs Fahrenheit—if you don't include the lower-level terms as well, so you'd better be sure you really want that. The form of the interaction is unusual: when $x_2=0$, the response is linearly related to $x_1$; as $x_2$ increases in absolute magnitude the relation becomes a tighter parabola, with a maximum or minimum depending on the sign of $\beta'_{12}$—it's hard to imagine this is what you had in mind. It's sometimes said such models violate the marginality principle: see Venables (1998), "Exegeses on linear models", S-Plus Users' Conference, Washington DC.

Collinearity is a red herring here—it's not a problem when it's a matter of how the predictors are defined (sometimes called structural collinearity) because you're never going to have to predict responses for predictor patterns that aren't exhibiting exactly the same collinearity.

NB You hear different views about centring & scaling predictors, discussion of which often drags in collinearity. It's important to realize that these concern ease of interpretation only (or occasionally accuracy of calculation—not an issue with algorithms used by modern software, as @Frank said).

share|improve this answer
Thanks for elaborate answer sir. But just to confirm again, i am not squaring x1x2 like you showed above, rather i am squaring the z-score of x1x2. Along with that in the regression model, i include x and x2 also along with certain other control variables. Does it change your viewpoint now? – user39496 Feb 17 '14 at 5:23
Thanks for pointing that out; I've altered the formula of the last model to reflect it. But it makes no difference to my viewpoint - which isn't a 'viewpoint', just the maths we learnt at school. Draw some graphs if the behaviour of the model isn't apparent, considering the consequences of what you're holding fixed & what you're allowing to vary. For instance, when $x_2=0$, the response varies linearly with $x_1$ - is that a reasonable restriction to impose before estimating any coefficient? – Scortchi Feb 17 '14 at 10:01

Centering before multiplying is a good idea but there is no need to take the z-score, much less to square it. Note that XM should be on the centered variables, so, really, more like $X_cM_c$ where the subscript indicates "centered".

share|improve this answer
Centering only helps in a very narrow sense, I think. It will not effect the estimates of the interaction parameters; it will only change the "main effects" which anyhow are not so interpretable in the presence of interaction. – Frank Harrell Feb 16 '14 at 15:44
Squaring the interaction may solve the collinearity problem but it also changes its identity into the interaction of $X^2$ and $M^2$. For that reason, I'd recommend not to call it "adjusting for interaction," simply because the model really is not doing that. – Penguin_Knight Feb 16 '14 at 16:04
Transforming XM with a z score will not substantively change the model. If you want to look at $X^2M^2$ then you can do so (although you should then probably include X and M and XM) – Peter Flom Feb 16 '14 at 16:26
Yes I disagree with the practice. If using modern software (e.g., R) centering doesn't help, and it makes interpretation more difficult. The Q-R decomposition is insensitive to extreme collinearity, and collinearity does not affect predicted values nor does it hurt hypothesis testing, at least for interesting hypotheses. – Frank Harrell Feb 16 '14 at 16:55
@user39496: To look at the $XM$ interaction you need to look at it, not at something else. Why do you think collinearity between $X$ & $XM$ or between $M$ & $XM$ is any sort of problem? Will you be needing to predict the response when $X=2$ & $M=3$ & $MX=10$? – Scortchi Feb 16 '14 at 17:00

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.