Currently, I want to check multi-collinearity among different categorical variables. FYI, I'm using 2 independent variables - category and division, and 1 independent variable - resolution_time_second as an example here.
First, the error message below occurred to me when I tried to perform VIF - Variance Inflation Factor (using R car::vif) on the glm() model.
Error in vif.default(fit) : there are aliased coefficients in the model
Then, I found out that I have encountered into so-called perfect multi-collinearity among variables in the dataset. By using the code below,I was able to find the "culprit" (variables) that is responsible for this issue.
formula <- as.formula(resolution_time_second~category+division)
fit <- glm(formula = formula, data = aeon_df, na.action = na.exclude)
print(summary(fit))
#output start
Call:
glm(formula = formula, data = aeon_df, na.action = na.exclude)
Deviance Residuals:
Min 1Q Median 3Q Max
-896572 -881875 -194707 12322 28406947
Coefficients: (3 not defined because of singularities)
Estimate Std. Error t value Pr(>|t|)
(Intercept) 425392 118409 3.593 0.000328 ***
categoryMerchandise - Hardline -214563 192958 -1.112 0.266167
categoryMerchandise - Softline -297032 148827 -1.996 0.045968 *
categoryService -188978 125610 -1.504 0.132472
divisionBeauty 62783 155165 0.405 0.685759
divisionCounter 660158 44207 14.933 < 2e-16 ***
divisionDaily & Dairy -193087 173113 -1.115 0.264702
divisionDelica -101288 183153 -0.553 0.580254
divisionDIY (Do-It-Yourself) 32526 277090 0.117 0.906556
divisionElectrical 20148 156872 0.128 0.897806
divisionGrocery -236761 133335 -1.776 0.075800 .
divisionHome Fashion -72694 172049 -0.423 0.672649
divisionHousehold 39243 180735 0.217 0.828111
divisionInnerwear -17123 169145 -0.101 0.919369
divisionKids 20360 111558 0.183 0.855183
divisionLadies -28405 131194 -0.217 0.828593
divisionMembership Services -41696 50180 -0.831 0.406027
divisionMen -26976 147591 -0.183 0.854976
divisionMultimedia -47570 183893 -0.259 0.795884
divisionNonfoods & HBC -270070 174931 -1.544 0.122638
divisionPerishable -219324 156251 -1.404 0.160434
divisionShoes, Bags & Accesorries NA NA NA NA
divisionSport -19197 248796 -0.077 0.938499
divisionStaff Related Issue NA NA NA NA
divisionStationery NA NA NA NA
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for gaussian family taken to be 2.089098e+12)
Null deviance: 4.3224e+16 on 19575 degrees of freedom
Residual deviance: 4.0850e+16 on 19554 degrees of freedom
(15146 observations deleted due to missingness)
AIC: 610905
Number of Fisher Scoring iterations: 2
#output end
or even easier (Notice that it's the same as the variables with "NA" coefficient)
attributes(alias(fit)$Complete)$dimnames[[1]]
#output start
[1] "divisionShoes, Bags & Accesorries" "divisionStaff Related Issue"
[3] "divisionStationery"
#output end
Hence, what I did is removing the categories with "NA" coefficient (aliased coefficient) from the categorical variable as shown below.
fit <- glm(formula, data=aeon_df, na.action = na.exclude,
subset = !division %in% c("Shoes, Bags & Accesorries","Staff Related Issue","Stationery"))
attributes(alias(fit)$Complete)$dimnames[[1]]
#output start
[1] "divisionMembership Services" "divisionMen"
[3] "divisionStationery"
#output end
However, I noticed the NEW categories with "NA" coefficient (aliased coefficient) still persisted in the model (as you can see from the output above). Even though I excluded the old ones, new ones still keep on occurring.
How can this happen and can I know is there a solution for this?
PS: I only show 2 categorical ivs for this example. I have more than 10 categorical ivs in my dataset.
