Collinearity in multivariate regression with huge amounts of data

Question

Take the following example:

I wish to predict physical performance as a function of height and weight. I already know weight negatively affects performance. Height also negatively affects performance, but I don't know if this is only because of the weight, or if there is a positive correlation between height and performance when disregarding the correlated weight. So, I measure the heights and weights of thousands of individuals, and measure their performance. I then model performance with multivariate regression: $p = (a * \text{height}) + (b * \text{weight}$). The estimated coefficients $\hat{a}$ and $\hat{b}$ turn out to be 0.2 and -1 respectively when using "lm" in R. The individual significances of these estimated coefficients are very high according to the t-tests (as is shown in a table when using the R command "summary" of a linear model made with the command "lm"), and their std. errors are very low. I therefore conclude that height is positively correlated with performance when disregarding the negative effects of weight. When is this conclusion a mistake?

To rephrase, if I have plenty of data and the std. error is very low for the estimated coefficients of both correlated variables (say $x_1$ and $x_2$) in the regression summary produced by the R command "summary", is there any reason (in general) why I should distrust the estimated coefficients for $x_1$ and $x_2$? I.e. can the collinearity mean that these estimated coefficients do not necessarily reflect the real coefficients even though the standard errors shown by the regression summary are very low?

Answer 1

According to Wikipedia, which in turn refers to Chatterjee, S.; Hadi, A. S.; Price, B. (2000). Regression Analysis by Example (Third ed.). John Wiley and Sons. ISBN 0-471-31946-5.,

So long as the underlying specification is correct, multicollinearity does not actually bias results; it just produces large standard errors in the related independent variables.

Hence, using the coefficients inferred from the correlated variables and interpreting them should not be problematic as long as the standard errors are low and the coefficients are not interpreted out of the context they were inferred (using the coefficients only makes sense if both coefficients are used together). Hence, there are no obvious problems with the conclusion drawn in the example in the question.

Answer 2

To add to the other excellent answers; "It depends" If multicollinearity is the only problem with the model, its effect would asymptotically vanish, when $n$ gets large enough. But, in practice, it might well be other problems with the model, which might interfere with multicolinearity!

Very specifically, if (some of) the regressor variables (the $x$'s) are measured with error, the problems caused by that will be amplified by multicollinearity, and I'm afraid that might happen in such a way that larger sample sizes will not help to much. I will try to add some simulations to this answer to illustrate/investigate that aspect.

Answer 3

As PeterFlom wrote, you can check ratio of min and max eigenvalues in the design matrix which tells you more. If CI is very big, then it means that inverting t(X)X will cause computational problems and estimates are not very reliable...