How to handle strongly correlated but not perfectly collinear dummies

Question

I am using several dummy variables in a GLM model implemented in R with a logit link function.

However, the coefficient of one of the dummy variables is not shown in the results with the warning that "Coefficients: (1 not defined because of singularities)".

When I check the correlations between the dummy variables with the cor() function, I see that 2 of them are indeed strongly, though not perfectly correlated (-0.6..).

I understand that one of the dummy variables could be removed if we had perfect collinearity. But how can this be handled in a situation where the correlation is "only" strong but not perfect without losing information (as I can not perfectly predict the correlated dummy)?

Answer 1

The note you see from R about a missing coefficient is not caused simply by correlations between columns (which are not in themselves a problem) but rather by the fact that your model matrix is not of full rank. Basically, your model is over-parametrized.

This is a problem caused by all the columns of your design matrix together. It is not a problem that can be analysed in terms of pairwise correlations alone. When all the columns of the model matrix are considered together, they are in fact perfectly collinear in one particular dimension, otherwise you would not get the message that you quote.

You will get a similar message as part of the output from summary(fit) whenever fit is a linear model or generalized linear model fit and there are one or more NA coefficients. It is not a warning or message in the formal R sense, but just a helpful note attached to the formatted table of coefficients. Looking down the table of coefficients will tell you which coefficient is NA.

Usually the message is a signal that you should reformulate your model matrix to avoid the over-parametrization. You might also find alias(fit) useful. Although the output from that function is brief, it attempts to show you exactly which coefficients are aliased with which others.

People have asked about similar messages many times on this help forum, for example

• Logistic regression "1 not defined because of singularities"
• Coefficients: (5 not defined because of singularities)
• How to deal with an error such as "Coefficients: 14 not defined because of singularities" in R?

If you provided more information about your data and glm call we might be able to give specific advice about your model matrix should be reformulated.