Use of collinearity in model building for regression

Question

How should I use concept of multicollinearity to decide which variables to consider for building a model from a given data. how should i know what should be the correlation value to be considered between the independent variables?

Answer 1

First, the title of the question is so broad that it would need a book or two to answer.

But in the text, you seem to be concerned mostly with collinearity and want to use correlations to weed out some variables. I'll just answer about that.

Correlations are not a good tool for diagnosing collinearity. You can have problematic collinearity with very low correlations. To see this, try this:

set.seed(1234)  #Sets a seed

x1 <- rnorm(100)
x2 <- rnorm(100)
x3 <- rnorm(100)
x4 <- rnorm(100)
x5 <- rnorm(100)
x6 <- rnorm(100)
x7 <- rnorm(100)
x8 <- rnorm(100)  
x9 <- rnorm(100)
x10 <- x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9

x <- cbind(x1, x2, x3, x4, x5, x6, x7, x8, x9, x10)

cor(x) #Most are very low

and yet, there is perfect collinearity. If we run

y <- x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9 + x10 + rnorm(100)

m1 <- lm(y ~ x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9 + x10)

the coefficient for x10 cannot be estimated.

If you want to diagnose collinearity, you should use condition indexes or (as a second choice) variance inflation factors.