5
$\begingroup$

Take the following example:

I wish to predict physical performance as a function of height and weight. I already know weight negatively affects performance. Height also negatively affects performance, but I don't know if this is only because of the weight, or if there is a positive correlation between height and performance when disregarding the correlated weight. So, I measure the heights and weights of thousands of individuals, and measure their performance. I then model performance with multivariate regression: $p = (a * \text{height}) + (b * \text{weight}$). The estimated coefficients $\hat{a}$ and $\hat{b}$ turn out to be 0.2 and -1 respectively when using "lm" in R. The individual significances of these estimated coefficients are very high according to the t-tests (as is shown in a table when using the R command "summary" of a linear model made with the command "lm"), and their std. errors are very low. I therefore conclude that height is positively correlated with performance when disregarding the negative effects of weight. When is this conclusion a mistake?

To rephrase, if I have plenty of data and the std. error is very low for the estimated coefficients of both correlated variables (say $x_1$ and $x_2$) in the regression summary produced by the R command "summary", is there any reason (in general) why I should distrust the estimated coefficients for $x_1$ and $x_2$? I.e. can the collinearity mean that these estimated coefficients do not necessarily reflect the real coefficients even though the standard errors shown by the regression summary are very low?

| cite | improve this question | |
$\endgroup$
  • $\begingroup$ Why not check for collinearity and see? $\endgroup$ – Peter Flom♦ Jan 6 '15 at 22:28
  • $\begingroup$ I already know these variables are highly correlated, but that is not the question here. The question concerns how correlated variables affect inference when there is a large amount of data $\endgroup$ – Datoraki Jan 6 '15 at 22:36
  • 2
    $\begingroup$ Correlation is not the same as colinear. Look at the condition indexes to see if there is a severe effect. $\endgroup$ – Peter Flom♦ Jan 6 '15 at 22:38
  • 1
    $\begingroup$ @PeterFlom (+1) for the useful CI remark.. $\endgroup$ – Analyst Jan 7 '15 at 8:51
  • $\begingroup$ @PeterFlom Thanks for pointing that out. I don't really think it answers the question though. Perhaps I was a bit unclear in my formulation $\endgroup$ – Datoraki Jan 12 '15 at 21:33
1
$\begingroup$

According to Wikipedia, which in turn refers to Chatterjee, S.; Hadi, A. S.; Price, B. (2000). Regression Analysis by Example (Third ed.). John Wiley and Sons. ISBN 0-471-31946-5.,

So long as the underlying specification is correct, multicollinearity does not actually bias results; it just produces large standard errors in the related independent variables.

Hence, using the coefficients inferred from the correlated variables and interpreting them should not be problematic as long as the standard errors are low and the coefficients are not interpreted out of the context they were inferred (using the coefficients only makes sense if both coefficients are used together). Hence, there are no obvious problems with the conclusion drawn in the example in the question.

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

To add to the other excellent answers; "It depends" If multicollinearity is the only problem with the model, its effect would asymptotically vanish, when $n$ gets large enough. But, in practice, it might well be other problems with the model, which might interfere with multicolinearity!

Very specifically, if (some of) the regressor variables (the $x$'s) are measured with error, the problems caused by that will be amplified by multicollinearity, and I'm afraid that might happen in such a way that larger sample sizes will not help to much. I will try to add some simulations to this answer to illustrate/investigate that aspect.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

As PeterFlom wrote, you can check ratio of min and max eigenvalues in the design matrix which tells you more. If CI is very big, then it means that inverting t(X)X will cause computational problems and estimates are not very reliable...

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.