149
$\begingroup$

I have a dataframe with many observations and many variables. Some of them are categorical (unordered) and the others are numerical.

I'm looking for associations between these variables. I've been able to compute correlation for numerical variables (Spearman's correlation) but :

  • I don't know how to measure correlation between unordered categorical variables.
  • I don't know how to measure correlation between unordered categorical variables and numerical variables.

Does anyone know how this could be done? If so, are there R functions implementing these methods?

Improve this question
$\endgroup$
1

6 Answers 6

Reset to default
135
$\begingroup$

It depends on what sense of a correlation you want. When you run the prototypical Pearson's product moment correlation, you get a measure of the strength of association and you get a test of the significance of that association. More typically however, the significance test and the measure of effect size differ.

Significance tests:

Effect size (strength of association):

Improve this answer
$\endgroup$
11
  • 7
    $\begingroup$ A very thorough explanation of the continuous vs. nominal case can be found here: Correlation between a nominal (IV) and a continuous (DV) variable. $\endgroup$
    – gung - Reinstate Monica
    Commented Dec 23, 2014 at 16:35
  • 3
    $\begingroup$ In the binary vs interval case there's the point-biserial correlation. $\endgroup$
    – Glen_b
    Commented Mar 12, 2015 at 22:54
  • $\begingroup$ What would be a better alternative to the chi-squared test for large samples? $\endgroup$
    – Waldir Leoncio
    Commented Jul 19, 2015 at 22:33
  • 2
    $\begingroup$ @WaldirLeoncio, yes but if the null is true, $p$ will be $<.05$ only $5\%$ of the time. That is the way it is supposed to work. If you want to know the magnitude of the effect as well as a test of the null, you may want to calculate Cramer's V along with the chi-squared test. $\endgroup$
    – gung - Reinstate Monica
    Commented Jul 20, 2015 at 12:58
  • 1
    $\begingroup$ As @gung pointed out, Correlation between a nominal (IV) and a continuous (DV) variable is an excellent link for how correlation for mixed variables can be done. Hmisc::rcorr does this beautifully and we can check it (for a mixed variables dataframe) as follows: as.data.frame(rcorr(as.matrix(data_frame),type = "pearson")$P) $\:$ as.data.frame(rcorr(as.matrix(data_frame),type = "pearson")$r) $\endgroup$
    – KarthikS
    Commented Oct 30, 2015 at 18:23
16
$\begingroup$

I've seen the following cheatsheet linked before:

https://stats.idre.ucla.edu/other/mult-pkg/whatstat/

It may be useful to you. It even has links to specific R libraries.

Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ The issue with this cheatsheet is it only concerns categorical / ordinal / interval variables. What I'm looking for is a method allowing me to use both numerical and categorical independant variables. $\endgroup$
    – Clément F
    Commented Jul 17, 2014 at 14:01
9
$\begingroup$

If you want a correlation matrix of categorical variables, you can use the following wrapper function (requiring the 'vcd' package):

catcorrm <- function(vars, dat) sapply(vars, function(y) sapply(vars, function(x) assocstats(table(dat[,x], dat[,y]))$cramer))

Where:

vars is a string vector of categorical variables you want to correlate

dat is a data.frame containing the variables

The result is a matrix of Cramer's V's.

Improve this answer
$\endgroup$
6
$\begingroup$

Depends on what you want to achieve. Let $X$ be the continuous, numerical variable and $K$ the (unordered) categorical variable. Then one possible approach is to assign numerical scores $t_i$ to each of the possible values of $K$, $i=1, \dots, p$. One possible criterion is to maximize the correlation between the $X$ and the scores $t_i$. With only one continuous and one categorical variable, this might not be very helpful, since the maximum correlation will always be one (to show that, and find some such scores, is an exercise in using Lagrange multipliers! With multiple variables, we try to find compromise scores for the categorical variables, maybe trying to maximize the multiple correlation $R^2$. Then the individual correlations will not more (except very special cases!) equal one.

Such an analysis can be seen as a generalization of multiple correspondence analysis, and is known under many names, such as canonical correlation analysis, homogeneity analysis, and many others. An implementation in R is in the homals package (on CRAN). googling for some of this names will give a wealth of information, there is a complete book: Albert Gifi, "Nonlinear Multivariate Analysis". Good luck!

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ (+1) Why use Lagrange multipliers? Just use the values of the continuous variable to score the categorical one. This also reveals why the max correlation is not necessarily $1$, which is attainable only when each category is paired with an unvarying set of values of the continuous variable. $\endgroup$
    – whuber ♦
    Commented Nov 17, 2014 at 10:04
  • $\begingroup$ I will edit to take into account this comment. $\endgroup$
    – kjetil b halvorsen ♦
    Commented Sep 9, 2016 at 13:41
2
$\begingroup$

I had a similar problem and I tried the Chi-squared-Test as suggested but I got very confused in assessing the P-Values against NULL Hypothesis.

I will explain how I interpreted categorical variables. I am not sure how relevant it is in your case. I had Response Variable Y and two Predictor Variables X1 and X2 where X2 being a categorical variable with two levels say 1 and 2. I was trying to fit a Linear Model

ols = lm(Y ~ X1 + X2, data=mydata)

But I wanted to understand how different level of X2 fits the above equation. I came across a R function by()

by(mydata,X2,function(x) summary(lm(Y~X1,data=x)))

What this code does is, it is trying to fit in Linear Model for each level of X2. This gave me all P-value and R-square, Residual standard error which I understand and can interpret.

Again I am not sure if this is what you want. I sort of compared different values of X2 in predicting Y.

Improve this answer
$\endgroup$
1
$\begingroup$

To measure the link strength between two categorical variable i would rather suggest the use of a cross tab with the chisquare stat

to measure the link strength between a numerical and a categorical variable you can use a mean comparison to see if it change significally from one category to an others

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ Welcome to the site, @DaSilvaLionel. You may notice that using the chi-squared test with two categorical variables has already been suggested above. $\endgroup$
    – gung - Reinstate Monica
    Commented Jun 15, 2016 at 16:28

Not the answer you're looking for? Browse other questions tagged or ask your own question.