Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I cannot differentiate clearly between "interaction" and "collinearity" in multiple linear regression. For me these terms are related but not the same.
I have searched the forum but could not find the answer. Please share your answer if you have. If you know there is an answer somewhere in the forum for this kind of question, please share the link.
An interaction may arise when considering the relationship among three or more variables, and describes a situation in which the simultaneous influence of two variables on a third is not additive. Most commonly, interactions are considered in the context of regression analyses.
The presence of interactions can have important implications for the interpretation of statistical models. If two variables of interest interact, the relationship between each of the interacting variables and a third "dependent variable" depends on the value of the other interacting variable. In practice, this makes it more difficult to predict the consequences of changing the value of a variable, particularly if the variables it interacts with are hard to measure or difficult to control.
Collinearity is a statistical phenomenon in which two or more predictor variables in a multiple regression model are highly correlated, meaning that one can be linearly predicted from the others with a non-trivial degree of accuracy. In this situation the coefficient estimates of the multiple regression may change erratically in response to small changes in the model or the data. Collinearity does not reduce the predictive power or reliability of the model as a whole, at least within the sample data themselves; it only affects calculations regarding individual predictors. That is, a multiple regression model with correlated predictors can indicate how well the entire bundle of predictors predicts the outcome variable, but it may not give valid results about any individual predictor, or about which predictors are redundant with respect to others.
Bottom line: Interactions don't imply collinearity and collinearity does not imply there are interactions.
Interaction terms can be introduced to your model to account for different effects when two independent variables combine in some interesting way. They are commonly used when categorical factors are present, e.g. to allow different rates of response of $Y$ (income) to a second factor $X_{2}$ (years of education) according to the categorical $X_{1}$ (gender, 0=male, 1= female). If we simply regress $Y = \beta_{0} + \beta_{1}X_{1} + \beta_{2}X_{2}$, the model only accounts for females earning a fixed amount more or less than males, with a separate term accounting for educational difference regardless of gender.
If we add a third interaction variable $X_{1}X_{2}$, so $Y = \beta_{0} + \beta_{1}X_{1} + \beta_{2}X_{2} + \beta_{3}(X_{1}X_{2})$, this third factor will be zero for males, but non-zero for females, thus representing the specific variable female years of education, and allowing the model to separately account for the effects this on income ($\beta_{2}$ becomes the gradient for rate of change of male income, and $\beta_{3}$ is an adjustment to the slope of income change for females).
This is an interaction, but it is not collinear, because it does not respond the same way as years of education across the full data set (it is zero for all points that are male). Thus, interaction is not the same as collinearity, and interaction can in fact be a useful element of a regression model.
More generally, the multiplication of two variables (both non-constant) is by definition non-linear, so while an interaction term created in this way may be related to the two component variables, it is not related linearly so collinearity is impossible.
To me, 'interaction' is a term to describe how the responder(Y) reacts to different levels or level combinations of predictors (Xs). Eg., whether Y is influenced by X1 and X2 in a 1+1>2 manner. It does not describe or indicate the dependencies between the predictors. While multicollinearity is a term that describes such relationships/dependencies between the predictors.
INTERACTION mostly applied in Two way Anova and tells the impact of two or more independent variable on a given variable i,e.each of the independent variables have the same impact on a given dependent variable.where as col-linearity indicates the correlation between two or more independent variable without including the dependent variable.
