Find correlation between two sets of categorical data

Question

The question is whether we can find a correlation between two sets of grades (categorical data).

Let’s say we have a dog competition and there are 1000 dogs participating.

There are two rounds of assessment:

• First round: dog owners give their assessment on the scale from A to C. Where A is excellent and C is bad. There are four criteria for assessment during both tours (behaviour etc).

• Second round: one judge gives his assessment of one dog based on the same assessment criteria as in round 1. however, grades vary from M - meeting expectation, E - exceeding expectation, B - Bellow expectation.

We understand that M is B, E is A and B is C.

After two rounds our table would look like:

| dog | round one | round two |
| --------------- | --------- | --------- |
| Dog1_criteria1  | A         | B         |
| Dog1_criteria2  | A         | E         |
| Dog1_criteria3  | A         | E         |
| Dog1_criteria4  | B         | M         |
| Dog2_criteria1  | A         | E         |
| Dog2_criteria2  | B         | M         |
| Dog2_criteria3  | A         | E         |
| Dog2_criteria4  | C         | B         |

....

How do we find a correlation between the two sets of answers?(regardless criteria and based on criteria).

Answer 1

You can find the correlation between two nominal categorical variables (or in a two-way contingency table) with Cramér's V. This is related to a chi-square test of independence. Note that this statistic is never negative.

However, a couple of notes. 1) In your example, the variables are actually ordinal in nature. Something like Kendall correlation should work here. 2) In your example, there is a nesting to the data. That is, each criterion is nested within a dog. Depending on what you want to know, you may want to use a more complex model, perhaps with ordinal regression.

The following example using Cramer's V in R can be run at rdrr.io/snippets/ without installing the software.

Here, Cramer's V is relatively large (0.49), but the chi-square test of association is not significant.

Data = read.table(header=TRUE, text="
Gender  Favorite.color
Male    Blue
Male    Green 
Male    Blue
Male    Purple              
Male    Blue              
Female  Red                  
Female  Purple                
Female  Blue                 
Female  Red
Female  Purple
Other   Blue
Other   Purple
")

Table = xtabs(~ Gender + Favorite.color, data=Data)

Table

   ###         Favorite.color
   ### Gender   Blue Green Purple Red
   ### Female    1     0      2   2
   ### Male      3     1      1   0
   ### Other     1     0      1   0

library(vcd)

assocstats(Table)

   ### Cramer's V        : 0.487

chisq.test(Table, simulate.p.value=TRUE, B=10000)

   ### Pearson's Chi-squared test with simulated
   ###   p-value (based on 10000 replicates)
   ###
   ### X-squared = 5.7, df = NA, p-value = 0.5749