I'm performing a multiple linear regression on 9 predictors and 1 response variable. It so happens that of the 9 predictors 3 of them have constant values, i.e. for each of these predictors, the values are constant and are the same for each sample.

I would expect that the resulting coefficients for these variables would be zero; however this is only the case for 2 of the three constant variables. In the third case the coefficient is non-zero even though the variable has zero variance in the model (I double-checked this). Should this be possible?

  • 1
    $\begingroup$ On the face of it, the non-zero coefficient shouldn't happen. In fact, why include these predictors in the model at all? They can't help. $\endgroup$ – Nick Cox Apr 4 '16 at 10:53

Sure – it just amounts to adding a constant to the model in a different way:

$$\beta^T X + c = \sum_i \beta_i x_i + c = \sum_{i \notin C} \beta_i x_i + \left( \sum_{i \in C} \beta_i x_i + c \right), $$

where $C$ is the set of constant predictors, so that the parenthesized term is a constant.

This will cause issues with multicollinearity, and if you're doing simple linear regression the distribution among the $\beta_i$ for constant predictors will be arbitrary, but it's still a well-defined model. If you do any regularization, though, these $\beta_i$ should all become zero.

  • $\begingroup$ Isn't t weird though that only one of the constant predictors has a non-zero value? Shouldn't they all have one? $\endgroup$ – Héctor van den Boorn Apr 4 '16 at 10:27
  • $\begingroup$ As @Dougal said 'the distribution ... will be arbitrary'. In this case your software is making that decision for you. I suspect if you run the same problem using different software it will do something different. $\endgroup$ – mdewey Apr 4 '16 at 15:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.