3
$\begingroup$

I am fitting a logistic regression for the response variable- 0 or 1. There are 15 explanatory variables- 10 are continuous and 5 are categorical with 3 levels each. I checked collinearity among the 10 continuous variables using correlation and they are okay. Using R, the glm function returns NA as the coefficient for one of the level of a categorical variable.

How can I fix this problem?

Please help.

$\endgroup$
3
  • $\begingroup$ NOTE: Although this is about R, the same issue arises in other programs (such as SAS) so I think this should be left open. $\endgroup$
    – Peter Flom
    Dec 31, 2018 at 13:11
  • $\begingroup$ Possible duplicate of Dummy Variable, Reference Group $\endgroup$ Dec 31, 2018 at 13:33
  • $\begingroup$ if the rank is less than the number of columns then what should we do? should we remove the variable showing NA values? $\endgroup$ Jan 17 at 9:51

1 Answer 1

6
$\begingroup$

This problem often indicates that you have a singular design matrix $X$. You can check that by seeing whether the rank of the cross-product $X^\top X$ equals the number of the columns of $X$.

This can easily be performed in R using

ncol(X) == qr(X)$rank

Here is an R-example with some simulated data

N <- 10
x <- rnorm(N)
z <- sample(c(1,2,3),N,replace=TRUE)
y <- sample(c(0,1),N,replace=TRUE)
data <- data.frame(y=y,x=x,z=as.factor(z))
model<-glm(y~x+z,data=data,family="binomial")
summary(model)

# Get model matrix ...
X <- model.matrix(~x+z,data=data)

# Get rank of model matrix
qr(X)$rank

# Get number of parameters of the model = number of columns of model matrix
ncol(X)

# See if model matrix has full rank
ncol(X) == qr(X)$rank
$\endgroup$
3
  • 5
    $\begingroup$ Good answer, but there's no need to run model.matrix(), qr() or ncol() as these quantities are already available as part of the glm fit object. The rank of the model matrix is available as model$rank and the number of parameters is length(model$coefficients). $\endgroup$ Dec 31, 2018 at 22:48
  • 1
    $\begingroup$ The question was how to fix the model. Would it be okay to remove the variables? $\endgroup$ Mar 11, 2022 at 23:52
  • 1
    $\begingroup$ Removing the coefficients will "fix" the model in the sense that it will no longer have NA coefficients. But if the rank is less than the number of columns, that says that the information in the NA columns is already being perfectly represented by the other columns. Why that is the case seems like a question worth answering. $\endgroup$
    – Kyle Pena
    Mar 14, 2022 at 15:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.