What to do with collinear variables

Question

Disclaimer: This is for a homework project.

I'm trying to come up with the best model for diamond prices, depending on several variables and I seem to have a pretty good model so far. However I have run into two variables that are obviously collinear:

>with(diamonds, cor(data.frame(Table, Depth, Carat.Weight)))
                   Table       Depth Carat.Weight
Table         1.00000000 -0.41035485   0.05237998
Depth        -0.41035485  1.00000000   0.01779489
Carat.Weight  0.05237998  0.01779489   1.00000000

Table and Depth are dependent on each other, but I still want to include them in my predictive model. I did some research on diamonds and found that Table and Depth are the length across the top and the distance from top to bottom tip of a diamond. Since these prices of diamonds seem to be related to beauty and beauty seems to be related proportions, I was going to include their ratio, say $\frac{Table}{Depth}$, to predict the prices. Is this standard procedure for dealing with collinear variables? If not, what is?

Edit: Here is a plot of Depth ~ Table:

Answer 1

Those variables are correlated.

The extent of linear association implied by that correlation matrix is not remotely high enough for the variables to be considered collinear.

In this case, I'd be quite happy to use all three of those variables for typical regression applications.

One way to detect multicollinearity is to check the Choleski decomposition of the correlation matrix - if there's multicollinearity there will be some diagonal elements that are close to zero. Here it is on your own correlation matrix:

> chol(co)
     [,1]       [,2]       [,3]
[1,]    1 -0.4103548 0.05237998
[2,]    0  0.9119259 0.04308384
[3,]    0  0.0000000 0.99769741

(The diagonal should always be positive, though some implementations can go slightly negative with the effect of accumulated truncation errors)

As you see, the smallest diagonal is 0.91, which is still a long way from zero.

By contrast here's some nearly collinear data:

> x<-data.frame(x1=rnorm(20),x2=rnorm(20),x3=rnorm(20))
> x$x4<-with(x,x1+x2+x3+rnorm(20,0,1e-4))
> chol(cor(x))
   x1         x2         x3           x4
x1  1 0.03243977 -0.3920567 3.295264e-01
x2  0 0.99947369  0.4056161 7.617940e-01
x3  0 0.00000000  0.8256919 5.577474e-01
x4  0 0.00000000  0.0000000 7.590116e-05   <------- close to 0.

Answer 2

Thought this diamond-cutting schematic might add insight to the Question. Can't add an image to a Comment so made it an answer....

PS. @PeterEllis's comment: The fact that "diamonds which are longer across the top are shorter from top to bottom" might make sense this way: Assume all uncut diamonds are roughly rectangular (say). Now the cutter must choose his cut with this bounding rectangle. That introduces the tradeoff. If both width and length increase you are going for larger diamonds. Possible but rarer and more expensive. Make sense?

Answer 3

Using ratios in linear regression should be avoided. Essentially, what you are saying is that, if a linear regression was done on those two variables, they would be linearly correlated with no intercept; this is obviously not the case. See: http://cscu.cornell.edu/news/statnews/stnews03.pdf

Also,they are measuring a latent variable- the size(volume or area) of the diamond. Have you considered converting your data to a surface area/volume measure rather than include both variables?

You should post a residual plot of that depth and table data. Your correlation between the two may be invalid anyways.

Answer 4

From the correlation its difficult to conclude if the Table and Width are indeed correlated. A coefficient close to +1/-1 would say they are collinear. It also depends on the sample size..if you have more data use it to confirm.

The standard procedure in dealing with collinear variables is to eliminate one of them...cos knowing one would determine the other.

Answer 5

What makes you think that table and depth cause collinearity in your model? From the correlation matrix alone it's hard to tell that these two variables will cause collinearity issues. What does a joint F test tell you about both variables' contribution to your model? As curious_cat mentioned the Pearson may not be the best measure of correlation when the relationship is not linear (perhaps a rank based measure?). VIF and tolerance may help quantify the degree of collinearity you may have.

I think your approach of using their ratio is appropriate (though not as a solution to collinearity). When I see the figure, I immediately thought of a common measure in health research which waist to hip ratio. Although, in this case is more akin to BMI (weight/height^2). If the ratio is readily interpretable and intuitive in your audience, I don't see a reason not to use it. However, you maybe able to use both variables in your model unless there is clear evidence of collinearity.