5
$\begingroup$

In econometric analysis in some cases, such as models with interaction terms, multicollinearity between independent variables may exist. In such cases, some researchers suggest "mean-centering" strategy (subtract mean from the variables which appear as main effects and constitute the interaction terms). Usually, in the literature this methodology is applied when the dependent variable is normally distributed (e.g., Paper). Does mean centering remove multicollinearity if the variables are not normally distributed? Why is normal distribution required?

$\endgroup$
9
  • 2
    $\begingroup$ Mean centering does not remove multicolinearity (the covariance is unchanged) $\endgroup$
    – Firebug
    Commented Jun 16, 2023 at 6:58
  • $\begingroup$ @Firebug It changes ... See the answer by Affine here: stats.stackexchange.com/questions/60476/… $\endgroup$
    – Sane
    Commented Jun 16, 2023 at 7:01
  • 1
    $\begingroup$ Footnote #12 from A caution regarding using rules of thumb for variance inflation factors might be helpful to answer your question. $\endgroup$ Commented Jun 16, 2023 at 7:03
  • 1
    $\begingroup$ Oh, I had missed the part about the interaction terms. My previous comment was wrong: the covariance between interaction terms and the original variables does change based on centering (this is easy to prove) $\endgroup$
    – Firebug
    Commented Jun 16, 2023 at 8:32
  • $\begingroup$ Multicollinearity is a property of the design matrix $X.$ It has nothing to do with how its components might have arisen and therefore any distributional assumptions are irrelevant. $\endgroup$
    – whuber
    Commented Jun 16, 2023 at 20:32

1 Answer 1

6
$\begingroup$

The correlation with interaction terms does not depend on having a normal distribution.

Here is a example based on sampling from two independent exponential distributions (so with positive means) where the correlation with the interaction terms is high, but is largely removed by subtracting the means. It also shows that the correlation between the samples themselves is unaffected by subtracting the means.

set.seed(2023)
X <- rexp(10^6, 1)
Y <- rexp(10^6, 2)
mean(X)
# 0.9984457
mean(Y)
# 0.5001369

cor(X, Y)
# -0.001121004
cor(X, X*Y)
# 0.5781138
cor(Y, X*Y)
# 0.5757716

cor(X-mean(X), Y-mean(X))
# -0.001121004
cor(X-mean(X), (X-mean(X))*(Y-mean(Y)))
# 0.002229947
cor(Y-mean(Y), (X-mean(X))*(Y-mean(Y)))
# -0.0002722061

You need to subtract the means before taking the product, as subtracting a constant at the end has no impact on the correlation. Compare these with the earlier results:

cor(X-mean(X), X*Y-mean(X*Y))
# 0.5781138
cor(X, (X-mean(X))*(Y-mean(Y)))
# 0.002229947
$\endgroup$
2
  • $\begingroup$ Thanks for this. So do you mean that regardless of distribution of the variables mean-centering strategy can be applied to remove multicollinearity? Please have a look at the answer of David B here: stats.stackexchange.com/questions/606803/… . He claims that "A common reason why centering fails to remove multicollinearity is when variables are skewed or have a high kurtosis." $\endgroup$
    – Sane
    Commented Jun 16, 2023 at 10:11
  • 3
    $\begingroup$ @Sane - in my example, the exponential distributions $X$ and $Y$ are right-skewed, as are $XY$ and $(X-\bar X)(Y-\bar Y)$ though the last of these has a slightly strange shaped distribution. So it may be affected by particular cases. Your link says "mean-centering your variables prior to computing the interaction term will sometimes (but not always) reduce multicollinearity" $\endgroup$
    – Henry
    Commented Jun 16, 2023 at 10:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.