1
$\begingroup$

I'd like to discuss the centering of continuous predictor variables in multiple linear regression with an interaction term for the sake of "relieving" multicollinearity.

I've read about multicollinearity in many sources, but for the points I bring up here, this paper by Afshartous and Preston (2011) should offer sufficient background/support: http://www.amstat.org/publications/jse/v19n3/afshartous.pdf

For simplicity, I'm considering a multiple regression model with one binary predictor, one continuous and the interaction between the two. This implies I'm not considering a polynomial model where the interaction term is a given predictor multiplied by itself.

Note: In this thread, I'm not interested in the improved interpretability of a model with centered predictors (although I am aware of that benefit).

Summary points:

  • In order to avoid model misspecification, we should assess whether or not an interaction term is appropriate for a given set of data (look at graphs and assess with the p-value from the relevant regression model)
  • However, inherent in an interaction term is often notable multicollinearity with the lower-order terms from which it was constructed
    • Because of this, it's often suggested we center the continuous predictor before fitting a model with an interaction model
  • In Afshartous and Preston (2011), they illustrate how the SE and thus test statistic (t) and thus p-value for the interaction term are not affected by centering (or lack thereof)

Now, consider the general approach to fitting a regression model of the sort described above:

  1. Fit the "full" model: Include both main effects and the interaction term. If the interaction is significant, this is the final model. Additionally, the individual "main effects" are not meaningful on their own (neither their p-values nor parameter estimates), as both variables are interpreted with respect to their involvement in the interaction.
  2. If, after fitting the "full" model the interaction is not significant, remove it and assess the "main effects" model.

And herein lies the point I'd like to discuss:

  • Since inference for the interaction term is not affected by multicollinearity, as described above, we may trust its p-value even when the continuous predictor has not been centered

    • As such, if the interaction is significant, we trust this is a "real" result and may go on to interpret the model as described in point (1) above, since we aren't concerned about the individual p-values from the main effects at this point anyway (each predictor is important because it's involved in an interaction and thus will remain in the model).

    • Similarly, if the p-value for the interaction term is not significant, we also trust this as a valid result and know that we should remove this term. Upon doing so, we have alleviated any issue with collinearity since the interaction term is no longer in the model, and thus we may trust the inference (p-values) for the main effects in the main effects-only model.

In summary, I feel the two points above indicate there is no motivation from an inferential standpoint to center a predictor in the described type of model, because, no matter what, it will always be correctly specified since the interaction term is not affected by collinearity. What I'm looking for is confirmation that my logic is correct.

Lastly, my other question is this: I understand that, inherently, the parameter estimates for the main effects will be different between a centered and uncentered model solely because you're estimating two different parameters (stated many times in this paper and makes sense intuitively). However, what I'm still uncertain about is: Relatively speaking, are the parameter estimates for the main effects "incorrect" when you don't center the continuous predictor and there's collinearity between the main effects and interaction?

In other words/Why this matters: Say I want to predict the value of the outcome for given values of the predictors (so I literally write out the equation and plug-in specified values of each X). Here, the parameter estimates for the (uncentered) predictors come into play because they're being used in the calculation, but are they "correct" on the uncentered scale? Or have they been affected by multicollinearity and thus are "off" by some amount?

Final summary: I think inference and model specification are not problematic when the predictor (as described here) has not been centered, but from a predictive standpoint, may we trust the main effects parameter estimates from an uncentered model where the main effects are presumably collinear with the interaction term?

$\endgroup$
  • $\begingroup$ I'm not sure I understand your concern. You state that you're "not interested in improved interpretability" of a model but are interested from a "predictive" POV re lack of centering. Ashfartous, et al, go to some length to document that the "main effects" are evaluated at different points for uncentered vs centered predictors thereby producing differential std errors that do not necessarily reflect multicollinearity. They also cite several sources that centering removes "unnecessary ill-conditioning," if it exists. Finally, several times they state that it is a best practice to center. $\endgroup$ – DJohnson Nov 2 '15 at 13:08
  • $\begingroup$ You also note that, in the presence of a significant interactions, that the main effects no longer matter since the model is now evaluating the predictors in the 2-D interaction space. If Y-hat -- the predictions -- are unaffected by centering, what's your concern? $\endgroup$ – DJohnson Nov 2 '15 at 13:10
  • $\begingroup$ 1/5: The motivation for this question is from a teaching standpoint. I'm introducing interactions to non-stat majors for the first time, and don't want to complicate the idea by considering centering simultaneously with interactions. However, I also don’t want to fit an interaction model with an uncentered predictor if, in general, collinearity is a problem, as then I’d be showing them an incorrect process. Namely, I don’t want interpolation (predicted values based on reasonable values of the predictors) to be incorrect due to collinearity affecting model specification/parameter estimates. $\endgroup$ – Meg Nov 2 '15 at 20:08
  • $\begingroup$ 2/5: Indeed, centering a model allows for interpretability of the intercept term when a value of 0 is not reasonable for the predictor (i.e., when trying to extrapolate to 0, since 0 is not within the range of our data). The reason I’m setting this aspect aside is because I do discuss interpolation/extrapolation with my students and let them know that sometimes the predicted intercept value just doesn’t make sense in context (it equates to unreasonable extrapolation). $\endgroup$ – Meg Nov 2 '15 at 20:09
  • $\begingroup$ 3/5: So, I’m not interested in increased interpretability of a centered model for the sake of being able to say, "When x_1 (binary predictor) is zero and for the mean value of x_2 (centered continuous predictor), the predicted value of y is..." vs. “When x_1 and x_2 are both zero…” (in an uncentered model). $\endgroup$ – Meg Nov 2 '15 at 20:09

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.