What is the degrees of freedom of a linear model when some of the variables sum to 1?

Question

If I have a linear model

$$ y_i = \beta_0 +\beta_1 X_{1i} + \beta_2X_{2i} + \beta_3X_{3i} + \beta_4X_{4i}+ \beta_5X_{5i}+\beta_6X_{6i} + \epsilon_i$$

the degrees of freedom is $n-(6+1)$ because I have intercept + six parameters.

However, I have a model where part of the model sums to 0, e.g. $X_{1i}+X_{2i}+X_{3i} = 0$ and $X_{4i}+X_{5i}+X_{6i} = 0$.

Since in this case when $\beta_1 X_{1i}$ and $\beta_2X_{2i}$ are estimated $\beta_3X_{3i}$ is not free anymore, does this change my degrees of freedom to be $n-5$?

Answer 1

You describe a strict linear dependence among your predictors, which is dangerous and won't provide a unique solution for all the coefficients. As you describe it you only have 4 linearly independent predictors, not 6.

If you are succeeding in fitting your data with this model, your software might be arbitrarily (maybe even silently) removing predictors from your model to allow a solution. For example, perhaps $X_{3t}$ and $X_{6t}$ and their coefficients were removed, as they were the last offending predictors entered into your model. Yes, then you are only fitting 4 predictor coefficients plus the intercept, for $n-5$ degrees of freedom.

The best idea here is not to specify a model where you know that the predictors are linearly dependent, as with linearly dependent predictors you won't necessarily get the same coefficients with different software or with a different order of predictors in your model or data set, even if the model fits at all. Unexpected near-dependence arising from predictor multicollinearity can be bad enough; there's no need to compound the problem with deliberate linear dependence.