Imagine a regression model where there is a continuous-valued response variable and three continuous-valued explanatory variables. For concreteness, imagine that we are interested in the effects of "Predictability," "Length," and "Frequenty," on the reading times "RT" of words.
Suppose further that we know that there is collinearity between "Frequency" and the other two explanatory variables. A way of dealing with this in the model is to residualize "Frequency," along the lines of the R code given (where the second line is the form of the model inputted in to a suitable regression algorithm):
r. <- function (formula, ...) rstandard(lm(formula, ...)) RT~Predictability+Length+r.(Frequency~Length+Predictability)
Now suppose that one of the the variables of interest was binary-valued. Let's say that we wanted to run a model with the form:
but we found that neither variable had a significant effect when both were included, but when a saturated systematic component,
Education*ReadsDaily was used, the model found highly significant results for both coefficients as well as the interaction term. When the two models are compared, the inclusion of the interaction term decreases the deviance by
16.7. When a VIF analysis is run on the model with an interaction terms, very high values are reported:
ReadDailyY Education ReadDailyY:Education 8.693957 4.266084 15.607665
The model without an interaction term, which fits poorly, has low values in the VIF analysis:
ReadDailyY Education 1.227842 1.227842
I think this means that there is multicollinearity between the two predictors (cf. Agresti 2007:138):
...models with several predictors often suffer from multicollinearity – correlations among predictors making it seem that no one variable is important when all the others are in the model. A variable may seem to have little effect because it overlaps considerably with other predictors in the model, itself being predicted well by the other predictors. Deleting such a redundant predictor can be helpful, for instance to reduce standard errors of other estimated effects.
We would, however, like to try to separate their effects on "RT" rather than combining the variables or discarding one, if at all possible, since the distinction between the two variables happens to be of theoretical importance. It would be possible to do something like the following in R with no error messages:
r. <- function(formula) rstandard(glm(family=binomial(),formula)) RT~Education+r.(ReadsDaily~Education)
The residualized "ReadsDaily" parameter in the model would then be continuous-valued. I suspect that this strategy for dealing with collinearity is methodologically suspect, but I do not know enough to give a reason for why or why not the strategy is to be dispreferred.
Agresti, A. (2007) An Introduction to Categorical Data Analysis. Wiley.