Does standardising independent variables reduce collinearity?

Question

I've come across a very good text on Bayes/MCMC. IT suggests that a standardisation of your independent variables will make an MCMC (Metropolis) algorithm more efficient, but also that it may reduce (multi)collinearity. Can that be true? Is this something I should be doing as standard.(Sorry).

Kruschke 2011, Doing Bayesian Data Analysis. (AP)

Thanks all for your time and helpful comments

R

edit: for example

     > data(longley)
     > cor.test(longley$Unemployed, longley$Armed.Forces)

Pearson's product-moment correlation

     data:  longley$Unemployed and longley$Armed.Forces 
     t = -0.6745, df = 14, p-value = 0.5109
     alternative hypothesis: true correlation is not equal to 0 
     95 percent confidence interval:
     -0.6187113  0.3489766 
     sample estimates:
      cor 
     -0.1774206 

     > standardise <- function(x) {(x-mean(x))/sd(x)}
     > cor.test(standardise(longley$Unemployed), standardise(longley$Armed.Forces))

Pearson's product-moment correlation

     data:  standardise(longley$Unemployed) and standardise(longley$Armed.Forces) 
     t = -0.6745, df = 14, p-value = 0.5109
      alternative hypothesis: true correlation is not equal to 0 
     95 percent confidence interval:
      -0.6187113  0.3489766 
      sample estimates:
       cor 
     -0.1774206 

This hasn't reduced the correlation or therefore the albeit limited linear dependence of vectors.

What's going on?

R

Answer 1

It doesn't change the collinearity between the main effects at all. Scaling doesn't either. Any linear transform won't do that. What it changes is the correlation between main effects and their interactions. Even if A and B are independent with a correlation of 0, the correlation between A, and A:B will be dependent upon scale factors.

Try this...

a <- rnorm(50)
b <- rnorm(50)
cor(a,b)

This should be near 0.

cor(a, a*b)

This should also be near 0.

However...

b <- rnorm(50, 5, 2)
cor(a,b)

This will again have a tiny correlation... but!!

cor(a, a*b)

Now that should have a substantial correlation which you can make go away by centring and/or standardizing. I generally go with just the centring.

Answer 2

Standardization does not affect the correlation between variables. They remain exactly the same. The correlation captures the synchronization of the direction of the variables. There is nothing in standardization that does change the direction of the variables.

If you want to eliminate multicollinearity between your variables, I suggest using Principal Component Analysis (PCA). As you know PCA is very effective in eliminating the multicollinearity problem. On the other hand PCA renders the combined variables (principal components P1, P2, etc...) rather opaque. A PCA model is always a lot more challenging to explain than a more traditional multivariate one.

Answer 3

Standardization is a common way to reduce collinearity. (You should be able to verify very quickly that it works by trying it out on a couple of pairs of variables.) Whether you do it routinely depends on how much of a problem collinearity is in your analyses.

Edit: I see I was in error. What standardizing does do, though, is reduce collinearity with product terms (interaction terms).