I have 20 independent variables that sum to one. I have 181 vectors so my X matrix is 181 by 20 so I do have an overdetermined system. However, when I run lm() in R with this data it gives me nonsensical data with equal coefficients for every single independent variable. Also, the intercept is -1*coefficient. Could someone please explain why linear regression doesn't work in this situation?
$\begingroup$
$\endgroup$
2
-
4$\begingroup$ I am afraid I only have time for a quick comment, but if you search for "compositional data analysis" you should quicky find some useful info. In short, if I have 3 variables that sum to 1, as soon as I know the value of 2 of them, I must then also know the value of the third...thus trying to estimate 3 independent "effects" is nonsensical. Also, please see this related question stats.stackexchange.com/questions/68944/… $\endgroup$– D L DahlyCommented Feb 19, 2014 at 8:04
-
$\begingroup$ Are you sure you did not simply regress one of the independent vectors against the rest? $\endgroup$– whuber ♦Commented Feb 19, 2014 at 15:25
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The independent variable corresponding to the intercept term is a column of 1's.
That's multicollinear with the sum of your x's. So you could add a constant to each of the coefficients on your x's and subtract the same constant from the intercept without changing the fit.
See item 1 here.
I'm surprised R didn't remove the last x, to be honest (check that).
(If they are all present, it makes me wonder whether every single row actually sums to 1.)
You'll need to either leave out one of the x's or leave out the intercept term. The meaning of the coefficients will change depending on which you do.
