Suppose i have the following model:

$Y = \alpha_0 + \alpha_1E_1 +\alpha_2E_2$, where $E_1$ indicates first group($1,2..n_1$) individuals, and $E_2$ indicates second group ($n_1,n_1 + 1,...n$)

I want to find OLS of the coefficients. But columns in my matrix X are not independent(because first column consists of one's and second with third columns are dependent from first column)

$(X^tX) = $$ \left( \begin{array}{ccc} n & n_1 & n_2 \\ n_1 & n_1 & 0 \\ n_2 & 0 & n_2 \end{array} \right)$

And because columns are not independent the inverse DNE.

The question is: how can I find OLS in this situation? Are my arguments correct?

Edit: I think, that we can test coefficients separately(if we consider the first group, that regression lools like: $Y = \alpha_0 + \alpha_1E_1$ and we can calculate coefficients without any problems. Is that correct way of solving the problem?


Usually when you have a categorical variable with $k$ levels, you insert $k-1$ dummy variables in your regression model, to guarantee the full rank of the design matrix in order to calculate the OLS estimates.

That's your case, you have 2 groups, you insert a dummy variable $E_1$ , when this is $0$ it's clear that the individual belongs to the other group.

So the model you wrote in the edit is the proper one for both groups:

if an individual $i$ belongs to the first group you are modeling

$E[Y_i] = \alpha_0 + \alpha_1*1 $

if it belongs to the second one

$E[Y_i] = \alpha_0 $

$E[Y_i]$ indicates expected value

  • $\begingroup$ Thank you for the comment! But I think, that if individual $i$ belongs to the second group the model is: $Y_i = \alpha_0 + \alpha_2*1$ in my case $\endgroup$ – Daniel Yefimov Jan 6 '17 at 11:59
  • $\begingroup$ @DanielYefimov there is no more $\alpha_2$, you don't need it in the model. If you put $E_2$ in the model then your design matrix is not invertible, and you can't find the $OLS$ estimates. You just need $E_1$ :). Think about it this way: if one individual does not belong to $E_1$ it certainly belongs to the other group, so I don't need another variable telling me if it belongs to the $2^{nd}$ group when I already know it belongs or not to the first one. $\endgroup$ – Tommaso Guerrini Jan 6 '17 at 12:02
  • $\begingroup$ I absolutely agree with you:) But the problem is, that the model in description is given to me, so i can't change it. So, i can conclude, that the inverse of my matrix $X$ DNE? And the solution of this problem is taking the general inverse of matrix? $\endgroup$ – Daniel Yefimov Jan 6 '17 at 12:14
  • 4
    $\begingroup$ If you cannot change it, then you should complain to the person who gave it to you, because the model, as explained by Tommaso, does not make sense. In that model, $\alpha_0$ gives you expected $Y$ for units neither in group 1 nor in group 2 (like in any regression: the expected $Y$ when all regressors are set to zero). Since nobody is in that category, you cannot identify all parameters $\alpha_0$, $\alpha_1$, $\alpha_2$ from your data. $\endgroup$ – Christoph Hanck Jan 6 '17 at 14:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.