Slightly related to previous questions by others, but more theoretical/hypothetical.

Is there an accepted phrase, or perhaps a decent argument to which you can kindly refer me, for including two independent variables that are somewhat collinear "because of the way the world is", when you still want to include the effects of both because you feel (for theoretical reasons), that both are important?

Let's say you are regressing to predict suicide rates (among 100 towns), and want to include both income poverty (town means) and percentage of households with piped water (per town), among other variables. Now these two variables run together, perhaps correlated at ".8". We understand why they are positively correlated, and we understand that they will present a pretty high VIF score. Let's say a VIF above 10. So, my reading suggests I should drop one of the two variables. But, surely, there is an argument to say they also measure subtly different things?

So - is there a standard way of explaining that one "knows that two variables will be collinear and correlated, because that's just how things are in real life", but that you retain both nonetheless because they are not, in reality cognates, and introduce important nuance into a regression?

  • 1
    $\begingroup$ It might be worth reconsidering and refining your understanding of the VIF guidelines. Those are rules of thumb meant primarily to guide those unfamiliar with collinearity. One would never drop a variable that theory says should be included merely because it has nonzero correlation with some other variable. From this perspective, no explanation is necessary or even expected unless you are teaching a first course in multiple regression. $\endgroup$ – whuber Nov 1 '19 at 13:23

Arguments against the inclusion of highly correlated variables, which lead to collinearity in the regression problem, are mathematical/statistical arguments. If two variables are highly correlated, it is harder for the model to seperate the explanatory contribution of these two variables between them. E.g. if you include gross income and net income into the model (which is also hard to justify with a "real-world argument"), then the model does not know, if all the effect comes from gross income, and zero effect from net income, or vice versa, or a mixture of both. This would be expressed in very wide confidence intervals for the associated parameters.

Nevertheless, it could be justified to include correlated variables to a certain extent. Even the inclusion of gross income as well as net income could be justified (it is unlikely though), if the amount of income tax has a very high explanatory power, then the model would form the difference of both values.

Essentially it is a trade-off between the negative effects of collinearity and the positive effect of the additional explanatory power of the additional variable.

Edith: of course in my example it would be best to directly include just the income tax as a variable.

| cite | improve this answer | |
  • $\begingroup$ Thank you, @ghlavin. Do you base your "it could be justified" on any precedent, experience or just common sense? $\endgroup$ – RandomForestRanger Nov 18 '19 at 13:20

As you already stated, there are mathematical/statistical arguments for excluding one of two colinear parameters. Thus, unless you have a strong argument you should exclude one.

Here an example, for a strong argument: Suppose, you are interested whether or not somebody is pregnant. Two input variables might be

  1. Does the person take the birth control pill?
  2. Is the person male or female?

I suppose, there exists a strong colinearity between these two input parameters. Nevertheless, it might be wise to include both parameters in your model.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.