1
$\begingroup$

If all of my coefficients in my logsitic model have really perfect t-statistics that all show sufficiently high significance but have two coefficients that have high VIF like 13-14 with sample size of 11000 for each independent variable, can I ignore the multicollinearity, given the way I propose to use the results?

I have a logistic regression model that has 6 independent variables where each independent variable and the dependent variable has the same sample size of 11000.

From this logistic regression I produce 6 predictions, based on changing the values of the independent variables. So let's say for prediction 1, I increase independent variable 1 by two units while all other independent variables are increased by only one unit. For prediction 2, I increase independent variable 2 by 2 units while all other independent variables are increased by only one unit and so on.

This makes a total of 6 different $y$ values which I need to use for my own purpose as below.

 y = a + b1*2+b2*1+b3*1+b4*1+b5*1+b6*1 + error
 y = a + b1*1+b2*2+b3*1+b4*1+b5*1+b6*1 + error
 ...
 y = a + b1*1+b2*1+b3*1+b4*1+b5*1+b6*2 + error

I do this for each independent variable and produce 6 different predicted $y$ values accordingly. As you can see above, a two-unit increase for different independent variable each time so it makes a total of 6 different $y$ values.

So I have 6 independent variables and I have 6 different predicted $y$ values, based on changing each independent variable separately. Of course all intercepts and coefficients are highly significant at 0.01 level or less.

Then my main objective is to use the numerical values of these 6 different predicted $y$ values as an input to a separate function to produce a numerical output, a "utility value."

This "utility value" is the one I need. I want to show how this final utility value from a different model differs from that produced by this logistic regression model, as a function of how $y$ changes with different emphasis on independent-variable increases up to two units.

$\endgroup$
  • 1
    $\begingroup$ This question reads remarkably like your other two questions at stats.stackexchange.com/questions/220189 and stats.stackexchange.com/questions/220214, both of which ask exactly the same thing: "can I ignore the multicollinearity." Could you explain how the answers you got do not address the situation? $\endgroup$ – whuber Jun 24 '16 at 17:46
  • 1
    $\begingroup$ It they are not correct, then please point that out in the previous threads rather than starting a new thread! In the present instance, it is difficult to understand what you are trying to do. This leads me to suspect that the problems with the previous threads could stem from a lack of clarity in the question. $\endgroup$ – whuber Jun 24 '16 at 18:14
  • 4
    $\begingroup$ Please take a break, calm down, and edit your question (ideally, your original question) to say more about the whole problem that you are trying to address, not just this particular aspect of your present solution as it relates to collinearity. That will make it easier for those who take the time to try to answer questions on this site to provide you with useful help. What general question are you trying to answer? Why do you have 12 "different" logistic regressions? What makes them different? What is the nature of the procedure into which you will be placing your values of y? $\endgroup$ – EdM Jun 24 '16 at 19:04
  • 2
    $\begingroup$ What you're doing doesn't make any sense to me. What is your ultimate "purpose"? I suspect there will be a better way of achieving it than what you're doing. Moreover, what are your variables & your data? What is the larger situation? Whether you can "ignore multicollinearity" depends on that information. $\endgroup$ – gung Jun 24 '16 at 19:28
  • 2
    $\begingroup$ Is the following what you are trying to do? First, run a multiple logistic regression with a binary outcome variable and 6 predictor variables. Second, use the coefficients from that logistic regression to make 6 predictions of new $y$ values; each prediction is based on setting values of each of the 6 predictors to 1, except for one predictor whose value is set to 2. Third, use those 6 predicted $y$ values as input to some further function. If so, answering your question will require more information on what you are trying to accomplish with that further, yet unspecified, function. $\endgroup$ – EdM Jun 24 '16 at 20:22
2
$\begingroup$

In this case it seems that multicollinearity might substantially affect the way that you will be able to interpret your ultimate results of "utility values" determined from the predicted probabilities, via your utility function. I would recommend that you look for a different approach to the overall problem that you are trying to address.

This is like an attempt to assess the relative importance among your independent variables (IVs), through their different effects on your utility values when you set the value of 1 IV to twice the values of the other 5, and perform this process for all 6 IVs. In general, assessing variable importance can be difficult, as discussed for example on this page. This particular way to evaluate your 6 IVs has some substantial problems.

In response to an earlier question on this matter, @DJohnson said:

Collinearity is a problem only if the model needs to enforce an assumption of the independence of predictors. It's known that the model coefficients usually aren't affected by collinearity, whereas std errors and t-values are ... Examples of where IV independence can be important include pricing models where the analyst needs to know what the impact of price changes are clear and free of the influence of the other predictors in the model.

In your use of this logistic regression model, it seems that you do need to "enforce an assumption of the independence of predictors" in order to make reliable interpretations of the utility values returned by the 6 predictions, each based on a change in a single predictor variable, that you propose.

If two IVs are highly correlated, it is unrealistic to expect in practice that you would ever find the value of one IV to double while its correlated partner IV stays unchanged. If your multicollinearity comes from more complicated relations among your IVs, then your setting the value of only one IV to double at a time is even more unrealistic. You do not know that the "impact" of any one IV is "clear and free of the influence of the other predictors in the model." In fact, you know the opposite: that some IVs are highly correlated.

The way that you are constructing the 6 predictions from your logistic regression model is something like extrapolating regression results beyond the bounds of the data: your are asking it to make predictions for circumstances unlike those from which the model was built, in terms of the combinations of IV values. So even if you could "ignore" multicollinearity for some purposes, like making predictions based on new real-world cases, it would seem unwise to ignore it here, given the way that you propose to use your model.

Furthermore, your proposed use of the logistic regression coefficients does not seem to take into account any of the uncertainties in the regression coefficient estimates, and it's not clear whether you are using techniques like cross validation or bootstrapping to evaluate the quality of your model building or its dependence on the peculiarities of your present data sample. There are probably better ways to accomplish your ultimate goal, if it is in some way related to evaluating the relative importance of your 6 IVs.

$\endgroup$
  • $\begingroup$ Maybe my last question to ask. I think multicollinearity between independent variable and depedent variable is fine, is this correct? $\endgroup$ – Eric Jun 25 '16 at 8:52
  • 2
    $\begingroup$ An independent variable that helps predict, linearly, a dependent variable will be correlated with the dependent variable. So that's desired. Technically that is not "multicollinearity," a term properly reserved for relations among independent variables. $\endgroup$ – EdM Jun 25 '16 at 11:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.