How should I use concept of multicollinearity to decide which variables to consider for building a model from a given data. how should i know what should be the correlation value to be considered between the independent variables?

  • $\begingroup$ I answered what I think a proper question on this topic would be. I also edited the question title to make it match the text. And I fixed the tags. $\endgroup$ – Peter Flom Jan 17 '20 at 11:55

First, the title of the question is so broad that it would need a book or two to answer.

But in the text, you seem to be concerned mostly with collinearity and want to use correlations to weed out some variables. I'll just answer about that.

Correlations are not a good tool for diagnosing collinearity. You can have problematic collinearity with very low correlations. To see this, try this:

set.seed(1234)  #Sets a seed

x1 <- rnorm(100)
x2 <- rnorm(100)
x3 <- rnorm(100)
x4 <- rnorm(100)
x5 <- rnorm(100)
x6 <- rnorm(100)
x7 <- rnorm(100)
x8 <- rnorm(100)  
x9 <- rnorm(100)
x10 <- x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9

x <- cbind(x1, x2, x3, x4, x5, x6, x7, x8, x9, x10)

cor(x) #Most are very low

and yet, there is perfect collinearity. If we run

y <- x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9 + x10 + rnorm(100)

m1 <- lm(y ~ x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9 + x10)

the coefficient for x10 cannot be estimated.

If you want to diagnose collinearity, you should use condition indexes or (as a second choice) variance inflation factors.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy