0
$\begingroup$

Let the following multilevel problem, where we try to predict the credit card balance of individuals $y_i$:

$$ x_{i 1}= \begin{cases}1 & \text { if } i \text { th person is from the South } \\ 0 & \text { if } i \text { th person is not from the South },\end{cases} $$

and the second should be

$$ x_{i 2}= \begin{cases}1 & \text { if } i \text { th person is from the West } \\ 0 & \text { if } i \text { th person is not from the West. }\end{cases} $$

Then both of these variables can be used in the regression equation, in order to obtain the model $$ y_i=\beta_0+\beta_1 x_{i 1}+\beta_2 x_{i 2}+\epsilon_i= \begin{cases}\beta_0+\beta_1+\epsilon_i & \text { if } i \text { th person is from the South } \\ \beta_0+\beta_2+\epsilon_i & \text { if } i \text { th person is from the West } \\ \beta_0+\epsilon_i & \text { if } i \text { th person is from the East. }\end{cases} $$

My problem now is to solve by linear regression under the problem that adding the intercept will add a lot of colinearity between the column vectors. One solution is to add one constraint:

$$ \beta_0 + \beta_1 + \beta_2 = 0 $$

And use restricted least square. But what is the interpretation of adding this constraint ? (Outside of the technical reason) I suppose that we cannot say anymore that $\beta_0$ can be interpreted as the average credit card balance for individuals from the east

$$ y_i=\beta_0+\beta_1 x_{i 1}+\beta_2 x_{i 2}+\epsilon_i= \begin{cases}-\beta_2+\epsilon_i & \text { if } i \text { th person is from the South } \\ -\beta_1+\epsilon_i & \text { if } i \text { th person is from the West } \\ -\beta_1 - \beta_2+\epsilon_i & \text { if } i \text { th person is from the East. }\end{cases} $$

$\endgroup$

1 Answer 1

1
$\begingroup$

If you use a single column for each dummy variable, this wouldn't induce collinearity. Collinearity arises when for $n$ categories you use precisely $n$ columns (one-hot way) so that sum of these columns produces a column of ones, which is a multiple of intercept column. In your case there would be combinations like

0 1
0 0
1 1
1 0

which don't sum up into a constant (unless those categories are linearly dependent in the first place).

Also note that imposing constraint $\beta_0 + \beta_1 + \beta_2 = 0$ and substituting for $\beta_0$ in the regression formula produces $y_i = \beta_1(x_{i1} - 1) + \beta_2(x_{i2} - 1) + \epsilon_i$ which amounts to re-coding categories as 0 and -1 and dropping intercept term.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.