Evidence that Collinearity Causes Predictors to be Insignificant

Question

Question: Is there any evidence that collinearity causes some predictors in this model to be insignificant?

Using R, I calculated the correlations of the predictor variables of a linear model.

         lcavol lweight  age  lbph   svi   lcp gleason pgg45
lcavol    1.00    0.19 0.22  0.03  0.54  0.68    0.43  0.43
lweight   0.19    1.00 0.31  0.43  0.11  0.10    0.00  0.05
age       0.22    0.31 1.00  0.35  0.12  0.13    0.27  0.28
lbph      0.03    0.43 0.35  1.00 -0.09 -0.01    0.08  0.08
svi       0.54    0.11 0.12 -0.09  1.00  0.67    0.32  0.46
lcp       0.68    0.10 0.13 -0.01  0.67  1.00    0.51  0.63
gleason   0.43    0.00 0.27  0.08  0.32  0.51    1.00  0.75
pgg45     0.43    0.05 0.28  0.08  0.46  0.63    0.75  1.00

I thought that there is a relatively strong correlation between lcp and lcavol, lcp and svi, lcp and gg45, gleason and gg45.

Would a correlation value of >0.5 be considered a strong correlation (ie: one of the variables would do a good job of representing the other)? How do we determine the minimum benchmark for when two variables have a strong correlation?

Answer 1

0.5 would not be considered high correlation in terms of the effect on standard errors. In my experience it only tends to become problematic at correlations quite close to 1.

A little simulation can demonstrate this.

> library(MASS)
> set.seed(1)
> N <- 20
> rho <- 0.5
> Sigma <- matrix(c(1, rho, rho, 1), 2, 2)
> u <- mvrnorm(n = N, c(10, 10), Sigma, empirical = TRUE)
> X1 <- u[, 1]
> X2 <- u[, 2]
> Y <- X1 + X2 + rnorm(N)
> lm( Y ~ X1 + X2 ) %>% summary()

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  -3.3380     2.1116  -1.581    0.132    
X1            1.1514     0.2104   5.472 4.13e-05 ***
X2            1.1963     0.2104   5.686 2.68e-05 ***

Now if we change the correlation to 0.95:

> set.seed(1)
> N <- 20
> rho <- 0.95
> Sigma <- matrix(c(1, rho, rho, 1), 2, 2)
> 
> u <- mvrnorm(n = N, c(10, 10), Sigma, empirical = TRUE)
> 
> X1 <- u[, 1]
> X2 <- u[, 2]
> 
> Y <- X1 + X2 + rnorm(N)
> 
> lm( Y ~ X1 + X2 ) %>% summary()

Coefficients:
            Estimate Std. Error t value Pr(>|t|)  
(Intercept)  -2.9106     1.8540  -1.570   0.1349  
X1            1.0814     0.5836   1.853   0.0813 .
X2            1.2235     0.5836   2.097   0.0513 .