1
$\begingroup$

I have predicted an ecological variable using OLS regression which showed the model accounts for more than 72% of the variance in the dependent variable (DV). However, I am also interested in which covariates have much impact on the DV. But I found that some of the independent variables are collinear making the Variance Inflation Factor > 10. What is the solution to the multicollinearity problem both for the continuous variables Temp and Vapor ( Drop one? ) and the categorical variables 1-8 which have very high VIF?

 Variable           Coeff.  Std Coeff.  VIF    Std Error    t      P -Value
 Constant          -0.228   0            0      0.086       -2.644  0.008
 Precipitation      <.001   0.151       2.688   <.001        8.541  0.0
 Solar Rad          0.002   0.343       2.836   <.001        18.939 <.001
 Temp              -0.116  -1.604       28.12   0.004       -28.11  0.0
 Water Stress       0.881   0.391       2.352   0.037        23.7   <.001
 Vapor Pressure     0.135   1.382       30.49   0.006        23.259 0.0
   1               -0.103   -0.109      52.086  0.074       -1.398  0.162
   2               -0.14    -0.048      6.49    0.079       -1.761  0.078
   3               -0.11    -0.048      10.007  0.077       -1.42   0.156
   4               -0.104   -0.234      236.288 0.073       -1.416  0.157
   5               -0.097   -0.242      285.244 0.073       -1.331  0.183
   6               -0.104   -0.09       35.067  0.074       -1.406  0.16
   8               -0.119   -0.261      221.361 0.073       -1.629  0.103
   ELEVATION        <.001   -0.115      3.917   <.001       -5.381  <.001

Condition Number: 59.833
Mean of Correlation Matrix: 0.221
1st Eigenvalue divided by m: 0.328
Improve this question
$\endgroup$
1
  • $\begingroup$ I would pay more attention to the standard errors of the coefficients than the VIF. After all, the main problem with multicolliniariy is that it increases standard errors. $\endgroup$
    – David Lane
    Commented Oct 7, 2017 at 15:54

1 Answer 1

Reset to default
1
$\begingroup$

For the categorical variable, you may collapse the variable and make it fewer levels as it is now. For instance, you may convert it to a dummy: level 2 and not level 2.

I think domain knowledge should come in to decide which variable to keep: temp or vapor pressure. Certainly, you may keep the one with smaller least squares (seems that you are minimizing the sum of squared errors to estimate the coefficients) or the one leading to smaller test error. If you only care about the prediction performance, you may keep both. If you want to interpret the coefficients, then you need to think which one makes more sense to keep. Does temperature leads to increasing vapor pressure? Then, I would say keep temperature but drop vapor pressure. When you are explaining the impact of temperature on your response variable, you need to explain the theory which may well involve vapor pressure.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for the explanation.To further explain my question- The dummy variables are actually landcovers ( forest, cropland,etc.) which are vital in determining the dependent variable ( C stock), so it would not give sense to reduce/collapse it since they are mutually exclusive.I was wondering( recently came across) if i am able to use Lasso regression for variable selection and prediction accuracy at the same time. $\endgroup$
    – Kaleab Woldemariam
    Commented Dec 30, 2015 at 18:52
  • $\begingroup$ As @Rwitch has more or less said, seeking predictive accuracy will not give you the same model as seeking to distinguish the causal effect of particular variables. Atheoretical prediction differs from causal explanation. An algorithm-based variable selection tool such as lasso will not help you with causality. $\endgroup$
    – rolando2
    Commented Jul 13, 2018 at 16:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.