Like many others I am seeking for a method to create a "correlation" like matrix for an unpredictable mix of variables which can contain unordered categorical ones as well.

While there are ideas of defaulting to different options depending on variable types for each pair of variables seemed complicated and using e.g. polycor relies on ordered categorical variables I felt going down the mutual information route would be more straight forward as suggested here.

So I made a little example to check whether it gives me what I want. Some categorical variables, some numerical ones and you can see that both cat1 vs. cat2 as well as num1 vs. num2 are "perfectly correlated" whereas cat1 vs. cat3 as well as num1 vs. num3 are "perfectly anti-correlated", at least when you read an order into the categorical variables based on their alphabetical sort order of values (which is not what I want, see title!):

library(infotheo)

cat1 <- c('a','b','c','a','b','c','a','b','c')
cat2 <- c('x','y','z','x','y','z','x','y','z')
cat3 <- c('z','y','x','z','y','x','z','y','x')
cat4 <- c('d','e','f','d','e','e','d','d','f')
cat5 <- c('c','e','y','b','f','x',NA,NA,NA)
num1 <- c(1,2,3,1,2,3,1,2,3)
num2 <- c(10,20,30,10,20,30,10,20,30)
num3 <- c(30,20,10,30,20,10,30,20,10)
num4 <- c(12,21,3,0,5,7,22,3,100)
num5 <- c(NA,NA,NA,NA,NA,NA,1,2,3)

df <- cbind.data.frame(cat1,cat2,cat3,cat4,cat5,num1,num2,num3,num4,num5)
df.num <- cbind.data.frame(num1,num2,num3,num4,num5)

cor.matrix.nats <- (mutinformation(discretize(df, disc="equalwidth")))
cor.matrix.nats <- cbind.data.frame(row.names(cor.matrix.nats),cor.matrix.nats)

cor.matrix.num.Spearman <- cor(df.num, method="spearman", use="pairwise.complete.obs")
cor.matrix.num.Spearman <- cbind.data.frame(row.names(cor.matrix.num.Spearman), cor.matrix.num.Spearman)

To my surprise the the mutual information cat1 vs. cat2 is a lot higher than cat1 vs. cat3 and in fact I observe the exact same difference for num1 vs. num2 compared to num1 vs. num3 (where Spearman produces the expected result). This leads me to believe that for some reason infotheo somehow uses alphabetical ordering to give unordered categorical variables a non-existing order.

While I checked the documentation and another great explanation on the interpretation of mutual information I still can't get my head around why order seems to play a role...

If anyone's got an explanation or a suggestion how to work around this, it would be highly appreciated!

Thanks Mark