In econometric analysis in some cases, such as models with interaction terms, multicollinearity between independent variables may exist. In such cases, some researchers suggest "mean-centering" strategy (subtract mean from the variables which appear as main effects and constitute the interaction terms). Usually, in the literature this methodology is applied when the dependent variable is normally distributed (e.g., Paper). Does mean centering remove multicollinearity if the variables are not normally distributed? Why is normal distribution required?
$\begingroup$
$\endgroup$
9
-
2$\begingroup$ Mean centering does not remove multicolinearity (the covariance is unchanged) $\endgroup$– FirebugJun 16 at 6:58
-
$\begingroup$ @Firebug It changes ... See the answer by Affine here: stats.stackexchange.com/questions/60476/… $\endgroup$– SaneJun 16 at 7:01
-
1$\begingroup$ Footnote #12 from A caution regarding using rules of thumb for variance inflation factors might be helpful to answer your question. $\endgroup$– medium-dimensionalJun 16 at 7:03
-
1$\begingroup$ Oh, I had missed the part about the interaction terms. My previous comment was wrong: the covariance between interaction terms and the original variables does change based on centering (this is easy to prove) $\endgroup$– FirebugJun 16 at 8:32
-
$\begingroup$ Multicollinearity is a property of the design matrix $X.$ It has nothing to do with how its components might have arisen and therefore any distributional assumptions are irrelevant. $\endgroup$– whuber ♦Jun 16 at 20:32
|
Show 4 more comments
1 Answer
$\begingroup$
$\endgroup$
2
The correlation with interaction terms does not depend on having a normal distribution.
Here is a example based on sampling from two independent exponential distributions (so with positive means) where the correlation with the interaction terms is high, but is largely removed by subtracting the means. It also shows that the correlation between the samples themselves is unaffected by subtracting the means.
set.seed(2023)
X <- rexp(10^6, 1)
Y <- rexp(10^6, 2)
mean(X)
# 0.9984457
mean(Y)
# 0.5001369
cor(X, Y)
# -0.001121004
cor(X, X*Y)
# 0.5781138
cor(Y, X*Y)
# 0.5757716
cor(X-mean(X), Y-mean(X))
# -0.001121004
cor(X-mean(X), (X-mean(X))*(Y-mean(Y)))
# 0.002229947
cor(Y-mean(Y), (X-mean(X))*(Y-mean(Y)))
# -0.0002722061
You need to subtract the means before taking the product, as subtracting a constant at the end has no impact on the correlation. Compare these with the earlier results:
cor(X-mean(X), X*Y-mean(X*Y))
# 0.5781138
cor(X, (X-mean(X))*(Y-mean(Y)))
# 0.002229947
-
$\begingroup$ Thanks for this. So do you mean that regardless of distribution of the variables mean-centering strategy can be applied to remove multicollinearity? Please have a look at the answer of David B here: stats.stackexchange.com/questions/606803/… . He claims that "A common reason why centering fails to remove multicollinearity is when variables are skewed or have a high kurtosis." $\endgroup$– SaneJun 16 at 10:11
-
3$\begingroup$ @Sane - in my example, the exponential distributions $X$ and $Y$ are right-skewed, as are $XY$ and $(X-\bar X)(Y-\bar Y)$ though the last of these has a slightly strange shaped distribution. So it may be affected by particular cases. Your link says "mean-centering your variables prior to computing the interaction term will sometimes (but not always) reduce multicollinearity" $\endgroup$– HenryJun 16 at 10:44