Is centering a valid solution for multicollinearity?

Question

Let's assume that $y = a + a_1x_1 + a_2x_2 + a_3x_3 + e$ where $x_1$ and $x_2$ both are indexes both range from $0-10$ where $0$ is the minimum and $10$ is the maximum. I found by applying VIF, CI and eigenvalues methods that $x_1$ and $x_2$ are collinear. Can these indexes be mean centered to solve the problem of multicollinearity?

Answer 1

When the model is additive and linear, centering has nothing to do with collinearity. Centering can only help when there are multiple terms per variable such as square or interaction terms. Even then, centering only helps in a way that doesn't matter to us, because centering does not impact the pooled multiple degree of freedom tests that are most relevant when there are multiple connected variables present in the model. For example, if a model contains $X$ and $X^2$, the most relevant test is the 2 d.f. test of association, which is completely unaffected by centering $X$. The next most relevant test is that of the effect of $X^2$ which again is completely unaffected by centering.

Answer 2

(An easy way to find out is to try it and check for multicollinearity using the same methods you had used to discover the multicollinearity the first time ;-)

No, unfortunately, centering $x_1$ and $x_2$ will not help you. When you have multicollinearity with just two variables, you have a (very strong) pairwise correlation between those two variables. Consider this example in R:

library(MASS)
set.seed(1)
X = mvrnorm(100, mu=c(30,30), Sigma=rbind(c(100,  97),
                                          c( 97, 100) ))
x1 = X[,1]
x2 = X[,2]
cor(x1, x2)
# [1] 0.9698819

Centering is just a linear transformation, so it will not change anything about the shapes of the distributions or the relationship between them. Instead, it just slides them in one direction or the other. To see this, let's try it with our data:

x1c = x1 - mean(x1)
x2c = x2 - mean(x2)
cor(x1c, x2c)
# [1] 0.9698819

The correlation is exactly the same. Here's what the new variables look like:

They look exactly the same too, except that they are now centered on $(0, 0)$.

To learn more about these topics, it may help you to read these CV threads:

• Centering: When should you center your data & when should you standardize?
• Multicollinearity: Is there an intuitive explanation why multicollinearity is a problem in linear regression?
• Remedies: Dealing with correlated regressors

Answer 3

When you ask if centering is a valid solution to the problem of multicollinearity, then I think it is helpful to discuss what the problem actually is. I say this because there is great disagreement about whether or not multicollinearity is "a problem" that needs a statistical solution. Many people, also many very well-established people, have very strong opinions on multicollinearity, which goes as far as to mock people who consider it a problem. The very best example is Goldberger who compared testing for multicollinearity with testing for "small sample size", which is obviously nonsense. Very good expositions can be found in Dave Giles' blog. See here and here for the Goldberger example.

Let me define what I understand under multicollinearity: one or more of your explanatory variables are correlated to some degree. What is the problem with that? Well, it can be shown that the variance of your estimator increases. Is this a problem that needs a solution? Well, from a meta-perspective, it is a desirable property. If your variables do not contain much independent information, then the variance of your estimator should reflect this. From a researcher's perspective, it is however often a problem because publication bias forces us to put stars into tables, and a high variance of the estimator implies low power, which is detrimental to finding signficant effects if effects are small or noisy. If this is the problem, then what you are looking for are ways to increase precision.

But stop right here! Note: if you do find effects, you can stop to consider multicollinearity a problem. Apparently, even if the independent information in your variables is limited, i.e. they are correlated, you are still able to detect the effects that you are looking for. So the "problem" has no consequence for you.

Now to your question: Does subtracting means from your data "solve collinearity"? One answer has already been given: the collinearity of said variables is not changed by subtracting constants. You can see this by asking yourself: does the covariance between the variables change? Well, since the covariance is defined as $Cov(x_i,x_j) = E[(x_i-E[x_i])(x_j-E[x_j])]$, or their sample analogues if you wish, then you see that adding or subtracting constants don't matter. Hence, centering has no effect on the collinearity of your explanatory variables.

Does centering improve your precision? In this case, we need to look at the variance-covarance matrix of your estimator and compare them. The problem is that it is difficult to compare: in the non-centered case, when an intercept is included in the model, you have a matrix with one more dimension (note here that I assume that you would skip the constant in the regression with centered variables). However, since there is no intercept anymore, the dependency on the estimate of your intercept of your other estimates is clearly removed (i.e. if you define the problem of collinearity as "(strong) dependence between regressors, as measured by the off-diagonal elements of the variance-covariance matrix", then the answer is more complicated than a simple "no"). In any case, it might be that the standard errors of your estimates appear lower, which means that the precision could have been improved by centering (might be interesting to simulate this to test this). Having said that, if you do a statistical test, you will need to adjust the degrees of freedom correctly, and then the apparent increase in precision will most likely be lost (I would be surprised if not).

If centering does not improve your precision in meaningful ways, what helps? You could consider merging highly correlated variables into one factor (if this makes sense in your application). Outlier removal also tends to help, as does GLM estimation etc (even though this is less widely applied nowadays).