0
$\begingroup$

I have used best subset regression with various selection criteria (e.g. Mallow's Cp and AIC) to compile a list of possible regression models to explain the response variable "Length". Below is the best model of the bunch,

Length = 5.770 + 0.1180 Max flow field width + 0.02691 time until ff + 0.1897 Duration of FF - 0.0465 Cooling dominate phase

however the coefficient for "cooling dominate phase" is negative in the regression but it has a positive correlation to the response variable. Here is the table correlation coefficients for the response and predictor variables in the model.

enter image description here

The VIF scores are all under 2 but being above 1 it suggests there is some degree of collinearity or multicollinearity happening. Here are the VIF's.

enter image description here

Can I still have confidence in the accuracy of the regression coefficients if the VIF's are low but the sign of a predictor variable coefficient is opposite from its correlation to the response variable?

$\endgroup$
  • $\begingroup$ "opposite from its correlation to the response variable?" I am not sure what this means. Your outputs do not show any information on the response variable. Are you concern about the -0.17? $\endgroup$ – Penguin_Knight Mar 12 '18 at 15:13
  • $\begingroup$ You seem undecided about what you want to do: if you're doing multiple regression, then it is pointless to compare its coefficient estimates to a single correlation coefficient (which is a simple regression). Multiple regression and simple regression are two different models and there is no necessary relationship between their results. $\endgroup$ – whuber Mar 12 '18 at 15:24
1
$\begingroup$

It's possible that the regression coefficients from a one-predictor regression and from a multiple-predictor regression being different. The signs do not need to agree, so aren't their magnitudes. So, what you see here is not abnormal.

While the VIF are not "high," the individual $R^2$ are not small. For example, take VIF 1.94:

$$VIF = \frac{1}{1 - R^2_{duration\ of\ FF}}$$

$$1/1.94= 1 - R^2_{duration\ of\ FF}$$

$$ R^2_{duration\ of\ FF}= 1 - 1/1.94$$

$$ R^2_{duration\ of\ FF}= 0.484$$

About 48% of the variability in "Duration of FF" can be explained by the other three independent predictors. Their correlations are not as weak as you think.

Can I still have confidence in the accuracy of the regression coefficients if the VIF's are low but the sign of a predictor variable coefficient is opposite from its correlation to the response variable?

Since you mentioned you selected many subsets so I guess your doing some sort of predictive model? If so, does this version give you the best prediction?

$\endgroup$
  • $\begingroup$ @ Penguin_Knight. Thanks for your reply. The tolerance of the predictor variables has one of the things I have been wrestling with. I am trying to come up with a predictive model but also would like to try and identify which set of variables best describe length and what that model is. The one listed in the question is not the best of the predictive models, but is the one that had the best balance between predictive power while still maintaining low VIF scores. $\endgroup$ – user3169598 Mar 12 '18 at 16:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.