1
$\begingroup$

My time series model suffers from multicollinearity between two independent variables.

When taking one of the variables out of the model I obtain a coefficient of 0.08 for variable 1 but when including both variables simultaneously the sign reverses and I get a coefficient of -0.33 for variable 1 and 0.45 for variable 2.

My professor said that if both variables were perfectly correlated the sum of both coefficients would be 0.12 and that 0.08 was therefore a sign for multicollinearity. Does anyone know a paper confirming what she said?

Also when running the regression exluding variable 1, I was expecting the coefficient for variable 2 to be somewhat similar to the coefficient of variable 1 (using the same logic as above, i.e. adding up the coefficients should yield a coefficient that is similar to running the regression alone) but the coefficient is very different (0.3). Can somepne explain why this might be the case?

Improve this question
$\endgroup$
7
  • $\begingroup$ What I miss to hear from you, @Wiebi, is whether 1) your variables (those referred to above) are of different lags, 2) whether they are of primary interest to the research question and 3) whether your model includes other variables than these two variables. $\endgroup$
    – Daniel
    Commented Jul 21, 2015 at 17:47
  • $\begingroup$ 1) no there are no lags in my model but 2) yes, unfortunately, they are of primary interest and procedures to curb multicollinearity (e.g. orthogonalization) change the interpretation of the coefficients so that they are no longer suitable for my purposes. In fact, I am rather looking for ways to better understand my data and explain why I get these results. $\endgroup$
    – Wiebi
    Commented Jul 21, 2015 at 17:55
  • $\begingroup$ You will probably have to open your statistical textbook for a decent answer. However, a starting point would be to consider the sign of the correlation coefficient or covariance (you can use the correlation coefficient as well as the standard deviation for each of the two variables to compute the covariance). This is because the beta of any variable in OLS, in part, is determined by its variance-covariance matrix, and because your model includes only two variables, it should be relatively easy to grasp why you observe the results you observe. $\endgroup$
    – Daniel
    Commented Jul 21, 2015 at 18:29
  • $\begingroup$ With limited information about your model, your data in general, and so forth, that is probably the best answer I can give you. $\endgroup$
    – Daniel
    Commented Jul 21, 2015 at 18:30
  • $\begingroup$ @Daniel I agree with your statement about generality. But based on the original question by Wiebi I believe my answer and subsequent comments are entirely correct. Also, for the record I think this does not qualify as an answer, but rather clarifying comments. It should be removed and the relevant pieces appended as comments to my answer (first half) and the original question (second half) for the sake of clarity for future readers. $\endgroup$
    – Gavin M. Jones
    Commented Jul 21, 2015 at 19:28

2 Answers 2

Reset to default
1
$\begingroup$

Why don't you check the variance inflation factors, if you are worried about multicollinearity? To me, your results indeed sound like multicollinearity, but there is no way to check for this by eyeballing, i.e. you cannot simply state that multicollinearity is an issue due to the behaviour of coefficients or standard deviations. Neither will your correlation coefficients as I don't know any person who agree what an acceptable correlation coefficient should be. You must check the variance inflation factors to judge whether multicollinearity is a serious issue in your model. Any decent statistical software has an option of inspection the variance inflation factors. Typically, these need to be below 5, at least for your interest variable. Otherwise, multicollinearity could be problematic. If you find that multicollinearity is, indeed, present in your model, then check the Wikipedia-page for Multicollinearity, in particular the "Remedies for multicollinearity"-section. There are many strategies for dealing with multicollinearity, once you have realised that multicollinearity is a problem in your statistical model.

Improve this answer
$\endgroup$
0
1
$\begingroup$

Strange behavior in coefficients and SE can be quite indicative of problems with multicollinearity, particularly when sign-switching of coefficients occurs upon adding new predictor variables (see here for a similar discussion).

If you think about it, we are actually fitting a plane through the cloud of data points when we have 2 predictors. If $x_1$ and $x_2$ are highly collinear, then the plane is fitting to a cloud of points that resembles a line or a tube instead of a dispersed cloud (as would be the case with less correlated variables). This results in instability and sign-switching!

I'm not sure about what your professor said regarding summing coefficients to as a sign of collinearity. However, I see three ways to diagnose multicollinearity in this case (two "art", one "science"):

  1. Plotting $x_1$ ~ $x_2$ and checking correlation coefficients. This should be your first step in exploring the relationship among predictors. (Although as another user alluded to, there is no hard cutoff for what is acceptable; this varies by field.)
  2. Sign-switching of coefficients when a new predictor is added, or large changes in SE. This is evidence of instability in the model which could be caused by collinearity. This might lead you to #3.
  3. Look at variance inflation factor (VIF) for a more quantitative measure of multicollinearity. (Although there is still some "art" to deciding where the cutoff is.)
Improve this answer
$\endgroup$
3
  • $\begingroup$ In fact, the bivariate correlation coeffcients are high (> 0.8) but VIFs are low (<5) - even when dividing the sample into subperiods. That is kind of why I am trying to look for other ways to think about the problem and how to intepret it. $\endgroup$
    – Wiebi
    Commented Jul 21, 2015 at 15:17
  • $\begingroup$ @Wiebi it is important as you go along to remember that there is an exponential relationship between VIF and and the correlation coefficient (r) see here: VIF = 1/(1 - r^2). Therefore (to the best of my knowledge) the correlation coefficient cannot be "high" but the VIF "low" at the same time; they are congruent. For example, a VIF of 10 indicates that r^2 = 0.9. A VIF of 5 indicates that r^2 = 0.8. $\endgroup$
    – Gavin M. Jones
    Commented Jul 21, 2015 at 17:07
  • $\begingroup$ When you are working with only 2 variables, the VIFs will be equal and the above comment will be true. $\endgroup$
    – Gavin M. Jones
    Commented Jul 21, 2015 at 18:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.