1
$\begingroup$

Currently, I want to check multi-collinearity among different categorical variables. FYI, I'm using 2 independent variables - category and division, and 1 independent variable - resolution_time_second as an example here.

First, the error message below occurred to me when I tried to perform VIF - Variance Inflation Factor (using R car::vif) on the glm() model.

Error in vif.default(fit) : there are aliased coefficients in the model

Then, I found out that I have encountered into so-called perfect multi-collinearity among variables in the dataset. By using the code below,I was able to find the "culprit" (variables) that is responsible for this issue.

formula <- as.formula(resolution_time_second~category+division)
fit <- glm(formula = formula, data = aeon_df, na.action = na.exclude)
print(summary(fit))

#output start
Call:
glm(formula = formula, data = aeon_df, na.action = na.exclude)

Deviance Residuals: 
     Min        1Q    Median        3Q       Max  
 -896572   -881875   -194707     12322  28406947  

Coefficients: (3 not defined because of singularities)
                                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)                         425392     118409   3.593 0.000328 ***
categoryMerchandise - Hardline     -214563     192958  -1.112 0.266167    
categoryMerchandise - Softline     -297032     148827  -1.996 0.045968 *  
categoryService                    -188978     125610  -1.504 0.132472    
divisionBeauty                       62783     155165   0.405 0.685759    
divisionCounter                     660158      44207  14.933  < 2e-16 ***
divisionDaily & Dairy              -193087     173113  -1.115 0.264702    
divisionDelica                     -101288     183153  -0.553 0.580254    
divisionDIY (Do-It-Yourself)         32526     277090   0.117 0.906556    
divisionElectrical                   20148     156872   0.128 0.897806    
divisionGrocery                    -236761     133335  -1.776 0.075800 .  
divisionHome Fashion                -72694     172049  -0.423 0.672649    
divisionHousehold                    39243     180735   0.217 0.828111    
divisionInnerwear                   -17123     169145  -0.101 0.919369    
divisionKids                         20360     111558   0.183 0.855183    
divisionLadies                      -28405     131194  -0.217 0.828593    
divisionMembership Services         -41696      50180  -0.831 0.406027    
divisionMen                         -26976     147591  -0.183 0.854976    
divisionMultimedia                  -47570     183893  -0.259 0.795884    
divisionNonfoods & HBC             -270070     174931  -1.544 0.122638    
divisionPerishable                 -219324     156251  -1.404 0.160434    
divisionShoes, Bags & Accesorries       NA         NA      NA       NA    
divisionSport                       -19197     248796  -0.077 0.938499    
divisionStaff Related Issue             NA         NA      NA       NA    
divisionStationery                      NA         NA      NA       NA    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for gaussian family taken to be 2.089098e+12)

   Null deviance: 4.3224e+16  on 19575  degrees of freedom
Residual deviance: 4.0850e+16  on 19554  degrees of freedom
  (15146 observations deleted due to missingness)
AIC: 610905

Number of Fisher Scoring iterations: 2
#output end

or even easier (Notice that it's the same as the variables with "NA" coefficient)

attributes(alias(fit)$Complete)$dimnames[[1]]

#output start
[1] "divisionShoes, Bags & Accesorries" "divisionStaff Related Issue"      
[3] "divisionStationery"
#output end

Hence, what I did is removing the categories with "NA" coefficient (aliased coefficient) from the categorical variable as shown below.

fit <- glm(formula, data=aeon_df, na.action = na.exclude,
       subset = !division %in% c("Shoes, Bags & Accesorries","Staff Related Issue","Stationery"))

attributes(alias(fit)$Complete)$dimnames[[1]]

#output start
[1] "divisionMembership Services" "divisionMen"                
[3] "divisionStationery"
#output end

However, I noticed the NEW categories with "NA" coefficient (aliased coefficient) still persisted in the model (as you can see from the output above). Even though I excluded the old ones, new ones still keep on occurring.

How can this happen and can I know is there a solution for this?

PS: I only show 2 categorical ivs for this example. I have more than 10 categorical ivs in my dataset.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.