This is a rather basic question regarding multicollinearity. Say that I want to use linear regression model to predict job happiness based on someones salary, since I assume that the relationship is not perfectly linear and the regression slope should differ in the low-paid and high-paid employees, I decide to use a model with interaction term for $\text{salary_group} \times \text{salary}$. On one hand, both variables are obviously correlated, on another, the model intuitively seems to make sense. Can this lead to any problems?
$\begingroup$
$\endgroup$
5
-
1$\begingroup$ If nothing else it will lead to problems of interpretability. Unless you have a strong reason to look at income from both perspectives (categories and the actual number), don't do it. Such a reason could be tax brackets that constitute a finite set of containers that salary falls in and that are treated differently. $\endgroup$– David ErnstCommented Sep 19, 2017 at 13:54
-
$\begingroup$ So there is not only $\text{salary}$ and $\text{salary_group}$ but also the interaction? That is even more complicated than your title suggests. $\endgroup$– Richard HardyCommented Sep 19, 2017 at 14:55
-
1$\begingroup$ @David It might be complicated, but it can work. One way to see this is to recognize that standard dummy-variable codings of the groups amount to an order-zero spline. If instead we were a bit more sophisticated and used a higher-order spline, we would recognize this proposal as a perfectly standard procedure (and readily interpretable). The interaction would look like an additional spline-like term contributing to the final nonlinear fit. Although unorthodox, there doesn't seem to anything to prohibit this approach outright. $\endgroup$– whuber ♦Commented Sep 19, 2017 at 17:03
-
$\begingroup$ @whuber Why not just use splines then if the goal is to better fit the nonlinear aspects of the data? (Assuming the usual square terms etc. don't achieve that either). I'm not saying the proposed model is always a bad idea, just that the proposed reason of "I assume the relationship is not perfectly linear" is a bad one. Tax brackets would be a more tangible reason for this model, perhaps there are others. $\endgroup$– David ErnstCommented Sep 19, 2017 at 18:18
-
$\begingroup$ @DavidErnst that's not the only reason, because of non-statistical reasons I need to focus on simple, preferably linear-regression, model. Obviously there are better approaches for such data, but I'm simply asking what could go wrong if I used linear regression like this. The example I provided is made-up, but reflects important aspects of the problem. $\endgroup$– TimCommented Sep 19, 2017 at 21:49
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
As noticed by whuber the model I described seems to be just a variation of regression using linear splines. Such models are described, e.g. by James H. Steiger on those slides or on those notes by Frank Harrell. The model can be thought as a piecewise regression
$$ E(Y|X) = \beta_0 + \beta_1 X + \beta_2 I_{\{X>a\}} + \beta_3 X \cdot I_{\{X>a\}} + \dots $$
or regression using linear splines
$$ E(Y|X) = \beta_0 + \beta_1 X + \beta_2 (X-a)_+ + \dots $$
where $a$ is a knot and $(u)_+$ takes value of $1$ if $u$ is positive and $0$ otherwise.
So it seems that this problem reduces to asking Is there a problem with multicollinearity and for splines regression?