# What to do with collinear variables

Disclaimer: This is for a homework project.

I'm trying to come up with the best model for diamond prices, depending on several variables and I seem to have a pretty good model so far. However I have run into two variables that are obviously collinear:

>with(diamonds, cor(data.frame(Table, Depth, Carat.Weight)))
Table       Depth Carat.Weight
Table         1.00000000 -0.41035485   0.05237998
Depth        -0.41035485  1.00000000   0.01779489
Carat.Weight  0.05237998  0.01779489   1.00000000


Table and Depth are dependent on each other, but I still want to include them in my predictive model. I did some research on diamonds and found that Table and Depth are the length across the top and the distance from top to bottom tip of a diamond. Since these prices of diamonds seem to be related to beauty and beauty seems to be related proportions, I was going to include their ratio, say $\frac{Table}{Depth}$, to predict the prices. Is this standard procedure for dealing with collinear variables? If not, what is?

Edit: Here is a plot of Depth ~ Table: • +1 for an interesting question but, no, this is definitely not a standard procedure for dealing with co-linear variables. Hopefully someone will give you a good answer on why not. It might still be a good thing to do in your case... – Peter Ellis Mar 14 '13 at 9:21
• The odd thing about this seems to be that the correlation of -0.4 suggests that diamonds which are longer across the top are shorter from top to bottom. This seems counter-intuitive - are sure it's correct? – Peter Ellis Mar 14 '13 at 9:23
• In general, $cor$ will only reveal linear dependence right? What if $Table$ and $Depth$ were non-linearly related? In that case, would there be some analog of colliniarity that poses a problem? Or is only a linear dependence a problem. – curious_cat Mar 14 '13 at 9:29
• @PeterEllis I was told that this is a real data set, yes. Looking at a plot of Depth~Table, it could be because the variance fans out for high Table values. – Mike Flynn Mar 14 '13 at 9:29

Those variables are correlated.

The extent of linear association implied by that correlation matrix is not remotely high enough for the variables to be considered collinear.

In this case, I'd be quite happy to use all three of those variables for typical regression applications.

One way to detect multicollinearity is to check the Choleski decomposition of the correlation matrix - if there's multicollinearity there will be some diagonal elements that are close to zero. Here it is on your own correlation matrix:

> chol(co)
[,1]       [,2]       [,3]
[1,]    1 -0.4103548 0.05237998
[2,]    0  0.9119259 0.04308384
[3,]    0  0.0000000 0.99769741


(The diagonal should always be positive, though some implementations can go slightly negative with the effect of accumulated truncation errors)

As you see, the smallest diagonal is 0.91, which is still a long way from zero.

By contrast here's some nearly collinear data:

> x<-data.frame(x1=rnorm(20),x2=rnorm(20),x3=rnorm(20))
> x\$x4<-with(x,x1+x2+x3+rnorm(20,0,1e-4))
> chol(cor(x))
x1         x2         x3           x4
x1  1 0.03243977 -0.3920567 3.295264e-01
x2  0 0.99947369  0.4056161 7.617940e-01
x3  0 0.00000000  0.8256919 5.577474e-01
x4  0 0.00000000  0.0000000 7.590116e-05   <------- close to 0.

• Thanks, I think I was simply confused between "correlated" and "collinear" – Mike Flynn Mar 27 '13 at 2:48 PS. @PeterEllis's comment: The fact that "diamonds which are longer across the top are shorter from top to bottom" might make sense this way: Assume all uncut diamonds are roughly rectangular (say). Now the cutter must choose his cut with this bounding rectangle. That introduces the tradeoff. If both width and length increase you are going for larger diamonds. Possible but rarer and more expensive. Make sense?

Using ratios in linear regression should be avoided. Essentially, what you are saying is that, if a linear regression was done on those two variables, they would be linearly correlated with no intercept; this is obviously not the case. See: http://cscu.cornell.edu/news/statnews/stnews03.pdf

Also,they are measuring a latent variable- the size(volume or area) of the diamond. Have you considered converting your data to a surface area/volume measure rather than include both variables?

You should post a residual plot of that depth and table data. Your correlation between the two may be invalid anyways.

From the correlation its difficult to conclude if the Table and Width are indeed correlated. A coefficient close to +1/-1 would say they are collinear. It also depends on the sample size..if you have more data use it to confirm.

The standard procedure in dealing with collinear variables is to eliminate one of them...cos knowing one would determine the other.

• I'm not sure I agree w/ this. The correlation is r=-.41, which is a reasonable magnitude for a correlation, I would think. Given the likely N (based on a glance at the plot) I would expect the r to be highly 'significant'. Whether or not Table & Depth are correlated enough to be called "collinear" is going to be a matter of definition (although I wouldn't call it problematic collinearity, either). Lastly, I would be wary of simply eliminating one of the variables unless the r were very close to |1| (e.g., ~.99)--I can't tell if that's what you mean. – gung - Reinstate Monica Mar 15 '13 at 1:24

What makes you think that table and depth cause collinearity in your model? From the correlation matrix alone it's hard to tell that these two variables will cause collinearity issues. What does a joint F test tell you about both variables' contribution to your model? As curious_cat mentioned the Pearson may not be the best measure of correlation when the relationship is not linear (perhaps a rank based measure?). VIF and tolerance may help quantify the degree of collinearity you may have.

I think your approach of using their ratio is appropriate (though not as a solution to collinearity). When I see the figure, I immediately thought of a common measure in health research which waist to hip ratio. Although, in this case is more akin to BMI (weight/height^2). If the ratio is readily interpretable and intuitive in your audience, I don't see a reason not to use it. However, you maybe able to use both variables in your model unless there is clear evidence of collinearity.