0
$\begingroup$

I want to check multicollinearity to avoid any redundancy in my database before doing the multinomial logistic regression with categorical dependent variable using R, knowing that the majority of my variables expressed as dichotomous and ordinal. Not the VIF method! Is there any other method that I can use before the regression?

$\endgroup$
3
  • 2
    $\begingroup$ pairs to explore graphically; corand cor.test to check correlation; PCA with prcomp to explore potential relationship/collinearity amongst the variables. $\endgroup$
    – Adam Quek
    Jun 29, 2016 at 1:34
  • $\begingroup$ I'am working with categorical data and factor variables I can't use the cor, cor.test and the PCA $\endgroup$
    – Asma
    Jun 29, 2016 at 10:46
  • 1
    $\begingroup$ Just create any arbitrary response you like--a constant will do--and run a least squares multiple regression. The software will automatically conduct multicollinearity checks of the independent variables. $\endgroup$
    – whuber
    Jun 29, 2016 at 13:25

1 Answer 1

1
$\begingroup$

You could transform your categorial variables into one-hot encoded dummy variables before doing what @Adam-Quek suggested:

# demo dummy data
d <- iris
head(d)

      Sepal.Length Sepal.Width Petal.Length Petal.Width Species
    1          5.1         3.5          1.4         0.2  setosa
    2          4.9         3.0          1.4         0.2  setosa
    3          4.7         3.2          1.3         0.2  setosa
    4          4.6         3.1          1.5         0.2  setosa
    5          5.0         3.6          1.4         0.2  setosa
    6          5.4         3.9          1.7         0.4  setosa

# one-hot encode dummy data
library(caret)
d2 <- data.frame(predict(dummyVars(~., d), d))
str(d2)

      Sepal.Length Sepal.Width Petal.Length Petal.Width Species.setosa Species.versicolor Species.virginica
    1          5.1         3.5          1.4         0.2              1                  0                 0
    2          4.9         3.0          1.4         0.2              1                  0                 0
    3          4.7         3.2          1.3         0.2              1                  0                 0
    4          4.6         3.1          1.5         0.2              1                  0                 0
    5          5.0         3.6          1.4         0.2              1                  0                 0
    6          5.4         3.9          1.7         0.4              1                  0                 0

Using such, you could use regular tools again (feature correlation, PCA, ...), like Adam suggested:

pairs(d2, upper.panel = NULL)

Pairs

library(corrplot)
corrplot(cor(d2), type = 'lower')

Correlation

pcs <- prcomp(d2, center = T, scale. = T, tol = 0.8)
print(pcs)

    Standard deviations:
    [1] 2.086732

    Rotation:
                            PC1
    Sepal.Length        0.4100521
    Sepal.Width        -0.2352425
    Petal.Length        0.4750053
    Petal.Width         0.4647101
    Species.setosa     -0.4508000
    Species.versicolor  0.1027178
    Species.virginica   0.3480821
$\endgroup$
3
  • $\begingroup$ Checks of correlation as advocated here are much weaker than checks of multicollinearity. Therein is the essential subtlety of the latter: it can exist even when every pair of variables is only weakly correlated. $\endgroup$
    – whuber
    Jun 29, 2016 at 13:23
  • $\begingroup$ I tried transforming my categorials variables into one-hot encoded dummy variables but the corrplot didn't work for me maybe because i have 3200 variables $\endgroup$
    – Asma
    Jun 29, 2016 at 13:51
  • $\begingroup$ In such case you could reduce the amount of variables first (e.g. using a correlation cap, see e.g. this answer). But the argument of @whuber is very valid, so you should also consider trying out what he mentioned in the the comment at your question. $\endgroup$ Jun 29, 2016 at 15:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.