123
$\begingroup$

I have a dataframe with many observations and many variables. Some of them are categorical (unordered) and the others are numerical.

I'm looking for associations between these variables. I've been able to compute correlation for numerical variables (Spearman's correlation) but :

  • I don't know how to measure correlation between unordered categorical variables.
  • I don't know how to measure correlation between unordered categorical variables and numerical variables.

Does anyone know how this could be done? If so, are there R functions implementing these methods?

$\endgroup$
113
$\begingroup$

It depends on what sense of a correlation you want. When you run the prototypical Pearson's product moment correlation, you get a measure of the strength of association and you get a test of the significance of that association. More typically however, the significance test and the measure of effect size differ.

Significance tests:

Effect size (strength of association):

$\endgroup$
  • 5
    $\begingroup$ A very thorough explanation of the continuous vs. nominal case can be found here: Correlation between a nominal (IV) and a continuous (DV) variable. $\endgroup$ – gung - Reinstate Monica Dec 23 '14 at 16:35
  • 3
    $\begingroup$ In the binary vs interval case there's the point-biserial correlation. $\endgroup$ – Glen_b Mar 12 '15 at 22:54
  • $\begingroup$ What would be a better alternative to the chi-squared test for large samples? $\endgroup$ – Waldir Leoncio Jul 19 '15 at 22:33
  • 2
    $\begingroup$ @WaldirLeoncio, yes but if the null is true, $p$ will be $<.05$ only $5\%$ of the time. That is the way it is supposed to work. If you want to know the magnitude of the effect as well as a test of the null, you may want to calculate Cramer's V along with the chi-squared test. $\endgroup$ – gung - Reinstate Monica Jul 20 '15 at 12:58
  • 1
    $\begingroup$ As @gung pointed out, Correlation between a nominal (IV) and a continuous (DV) variable is an excellent link for how correlation for mixed variables can be done. Hmisc::rcorr does this beautifully and we can check it (for a mixed variables dataframe) as follows: as.data.frame(rcorr(as.matrix(data_frame),type = "pearson")$P) $\:$ as.data.frame(rcorr(as.matrix(data_frame),type = "pearson")$r) $\endgroup$ – KarthikS Oct 30 '15 at 18:23
15
$\begingroup$

I've seen the following cheatsheet linked before:

https://stats.idre.ucla.edu/other/mult-pkg/whatstat/

It may be useful to you. It even has links to specific R libraries.

$\endgroup$
  • 3
    $\begingroup$ The issue with this cheatsheet is it only concerns categorical / ordinal / interval variables. What I'm looking for is a method allowing me to use both numerical and categorical independant variables. $\endgroup$ – Clément F Jul 17 '14 at 14:01
7
$\begingroup$

If you want a correlation matrix of categorical variables, you can use the following wrapper function (requiring the 'vcd' package):

catcorrm <- function(vars, dat) sapply(vars, function(y) sapply(vars, function(x) assocstats(table(dat[,x], dat[,y]))$cramer))

Where:

vars is a string vector of categorical variables you want to correlate

dat is a data.frame containing the variables

The result is a matrix of Cramer's V's.

$\endgroup$
6
$\begingroup$

Depends on what you want to achieve. Let $X$ be the continuous, numerical variable and $K$ the (unordered) categorical variable. Then one possible approach is to assign numerical scores $t_i$ to each of the possible values of $K$, $i=1, \dots, p$. One possible criterion is to maximize the correlation between the $X$ and the scores $t_i$. With only one continuous and one categorical variable, this might not be very helpful, since the maximum correlation will always be one (to show that, and find some such scores, is an exercise in using Lagrange multipliers! With multiple variables, we try to find compromise scores for the categorical variables, maybe trying to maximize the multiple correlation $R^2$. Then the individual correlations will not more (except very special cases!) equal one.

Such an analysis can be seen as a generalization of multiple correspondence analysis, and is known under many names, such as canonical correlation analysis, homogeneity analysis, and many others. An implementation in R is in the homals package (on CRAN). googling for some of this names will give a wealth of information, there is a complete book: Albert Gifi, "Nonlinear Multivariate Analysis". Good luck!

$\endgroup$
  • 1
    $\begingroup$ (+1) Why use Lagrange multipliers? Just use the values of the continuous variable to score the categorical one. This also reveals why the max correlation is not necessarily $1$, which is attainable only when each category is paired with an unvarying set of values of the continuous variable. $\endgroup$ – whuber Nov 17 '14 at 10:04
  • $\begingroup$ I will edit to take into account this comment. $\endgroup$ – kjetil b halvorsen Sep 9 '16 at 13:41
2
$\begingroup$

I had a similar problem and I tried the Chi-squared-Test as suggested but I got very confused in assessing the P-Values against NULL Hypothesis.

I will explain how I interpreted categorical variables. I am not sure how relevant it is in your case. I had Response Variable Y and two Predictor Variables X1 and X2 where X2 being a categorical variable with two levels say 1 and 2. I was trying to fit a Linear Model

ols = lm(Y ~ X1 + X2, data=mydata)

But I wanted to understand how different level of X2 fits the above equation. I came across a R function by()

by(mydata,X2,function(x) summary(lm(Y~X1,data=x)))

What this code does is, it is trying to fit in Linear Model for each level of X2. This gave me all P-value and R-square, Residual standard error which I understand and can interpret.

Again I am not sure if this is what you want. I sort of compared different values of X2 in predicting Y.

$\endgroup$
1
$\begingroup$

To measure the link strength between two categorical variable i would rather suggest the use of a cross tab with the chisquare stat

to measure the link strength between a numerical and a categorical variable you can use a mean comparison to see if it change significally from one category to an others

$\endgroup$
  • 2
    $\begingroup$ Welcome to the site, @DaSilvaLionel. You may notice that using the chi-squared test with two categorical variables has already been suggested above. $\endgroup$ – gung - Reinstate Monica Jun 15 '16 at 16:28

protected by whuber Sep 30 '16 at 4:19

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.