The question is whether we can find a correlation between two sets of grades (categorical data).

Let’s say we have a dog competition and there are 1000 dogs participating.

There are two rounds of assessment:

  • First round: dog owners give their assessment on the scale from A to C. Where A is excellent and C is bad. There are four criteria for assessment during both tours (behaviour etc).

  • Second round: one judge gives his assessment of one dog based on the same assessment criteria as in round 1. however, grades vary from M - meeting expectation, E - exceeding expectation, B - Bellow expectation.

We understand that M is B, E is A and B is C.

After two rounds our table would look like:

| dog | round one | round two |
| --------------- | --------- | --------- |
| Dog1_criteria1  | A         | B         |
| Dog1_criteria2  | A         | E         |
| Dog1_criteria3  | A         | E         |
| Dog1_criteria4  | B         | M         |
| Dog2_criteria1  | A         | E         |
| Dog2_criteria2  | B         | M         |
| Dog2_criteria3  | A         | E         |
| Dog2_criteria4  | C         | B         |


How do we find a correlation between the two sets of answers?(regardless criteria and based on criteria).


You can find the correlation between two nominal categorical variables (or in a two-way contingency table) with Cramér's V. This is related to a chi-square test of independence. Note that this statistic is never negative.

However, a couple of notes. 1) In your example, the variables are actually ordinal in nature. Something like Kendall correlation should work here. 2) In your example, there is a nesting to the data. That is, each criterion is nested within a dog. Depending on what you want to know, you may want to use a more complex model, perhaps with ordinal regression.

The following example using Cramer's V in R can be run at rdrr.io/snippets/ without installing the software.

Here, Cramer's V is relatively large (0.49), but the chi-square test of association is not significant.

Data = read.table(header=TRUE, text="
Gender  Favorite.color
Male    Blue
Male    Green 
Male    Blue
Male    Purple              
Male    Blue              
Female  Red                  
Female  Purple                
Female  Blue                 
Female  Red
Female  Purple
Other   Blue
Other   Purple

Table = xtabs(~ Gender + Favorite.color, data=Data)


   ###         Favorite.color
   ### Gender   Blue Green Purple Red
   ### Female    1     0      2   2
   ### Male      3     1      1   0
   ### Other     1     0      1   0



   ### Cramer's V        : 0.487

chisq.test(Table, simulate.p.value=TRUE, B=10000)

   ### Pearson's Chi-squared test with simulated
   ###   p-value (based on 10000 replicates)
   ### X-squared = 5.7, df = NA, p-value = 0.5749

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.