How to check the correlation between categorical and numeric independent variable in R? [duplicate]

Question

Just wondering if i need to check correlation between categorical and numeric independent variable in R, is there any specific package available in R. Or should i just find the correlation between the numerical independent variable?

Answer 1

There are several ways to determine correlation between a categorical and a continuous variable. However, I found only one way to calculate a 'correlation coefficient', and that only works if your categorical variable is dichotomous.

If your categorical variable is dichotomous (only two values), then you can use the point-biserial correlation. There is a function to do this in the ltm package.

library(ltm)
# weakly correlated example
set.seed(123)
x <- rnorm(100)
y <- factor(sample(c("A", "B"), 100, replace = TRUE))
biserial.cor(x, y)
[1] -0.07914586 
# strongly correlated example
biserial.cor(mtcars$mpg, mtcars$am)
[1] -0.5998324

You could do a logistic regression and use various evaluations of it (accuracy, etc.) in place of a correlation coefficient. Again, this works best if your categorical variable is dichotomous.

# weakly correlated
set.seed(123)
x <- rnorm(100)
y <- factor(sample(c("A", "B"), 100, replace = TRUE))
logit <- glm(y ~ x, family = "binomial")
# Accuracy
sum(round(predict(logit, type = "response")) == as.numeric(y)) / length(y)
[1] 0.15
# Sensitivity
sum(round(predict(logit, type = "response")) == as.numeric(y) & as.numeric(y) == 1) /
                         sum(as.numeric(y))
[1] 0.1013514
# Precision
sum(round(predict(logit, type = "response")) == as.numeric(y) & as.numeric(y) == 1) /
                         sum(round(predict(logit, type = "response") == 1))
[1] Inf
enter code here
# strongly correlated
mt_logit <- glm(am~mpg, data = mtcars, family = "binomial")
mt_pred <- round(predict(mt_logit, type = "response"))
# Accuracy
sum(mt_pred == mtcars$am) / nrow(mtcars)
[1] 0.75
# Sensitivity
sum(mt_pred == mtcars$am & mtcars$am == 1) /
                         sum(mtcars$am)
[1] 0.5384615
# Precision
sum(mt_pred == mtcars$am & mtcars$am == 1) /
                         sum(mt_pred == 1)
[1] 0.7777778

Again, if your categorical data is dichotomous, you could do the two-sample Wilcoxon rank sum test. The wilcox.test() function is available in base R. This is a non-parametric variation on the ANOVA.

# weakly correlated
set.seed(123)
x <- rnorm(100)
y <- factor(sample(c("A", "B"), 100, replace = TRUE))
df <- data.frame(x = x, y = y)
wt <- wilcox.test(df$x[which(df$y == "A")], df$x[which(df$y == "B")])
    Wilcoxon rank sum test with continuity correction

data:  df$x[which(df$y == "A")] and df$x[which(df$y == "B")]
W = 1243, p-value = 0.9752
alternative hypothesis: true location shift is not equal to 0

# strongly correlated
wilcox.test(mtcars$mpg[which(mtcars$am == 1)], 
            mtcars$mpg[which(mtcars$am == 0)], exact = FALSE) # exact = FALSE because there are ties
    Wilcoxon rank sum test with continuity correction

data:  mtcars$mpg[which(mtcars$am == 1)] and mtcars$mpg[which(mtcars$am == 0)]
W = 205, p-value = 0.001871
alternative hypothesis: true location shift is not equal to 0

You could also just do an ANOVA on your logit model from earlier.

# weakly correlated
anova(logit)
    Analysis of Deviance Table

Model: binomial, link: logit

Response: y

Terms added sequentially (first to last)


     Df Deviance Resid. Df Resid. Dev
NULL                    99     138.47
x     1  0.62819        98     137.84

# strongly correlated
anova(mt_logit)
Analysis of Deviance Table

Model: binomial, link: logit

Response: am

Terms added sequentially (first to last)


     Df Deviance Resid. Df Resid. Dev
NULL                    31     43.230
mpg   1   13.555        30     29.675

If your categorical variable is not dichotomous, you can use the Kruskal-Wallis test.

# weakly correlated
set.seed(123)
x <- rnorm(100)
y <- factor(sample(c("A", "B", "C"), 100, replace = TRUE))
kruskal.test(x~y)
    Kruskal-Wallis rank sum test

data:  x by y
Kruskal-Wallis chi-squared = 0.62986, df = 2, p-value = 0.7298

# strongly correlated
kruskal.test(mpg ~ cyl, data = mtcars)
    Kruskal-Wallis rank sum test

data:  mpg by cyl
Kruskal-Wallis chi-squared = 25.746, df = 2, p-value = 2.566e-06

Finally, you can just inspect your data visually using some boxplots. If your data are weakly correlated, there will be a lot of overlap between the boxes.

library(ggplot2)
# weakly correlated
set.seed(123)
y <- rnorm(100)
x <- factor(sample(c("A", "B", "C"), 100, replace = TRUE))
df <- data.frame(x = x, y = y)
ggplot(df) + geom_boxplot(aes(x, y))   

# strongly correlated
ggplot(mtcars) + geom_boxplot(aes(x = factor(cyl), y = mpg))