Visualizing interrater disagreement I am trying to visualize the process of two raters who have each rated the same dataset. Each row (line in the plot) represent a heart beat (or an error) on an ecg. Every disagreement was discussed and commen rating was agreed upon. The agreed ratings are stored in the variable common.
I want to show frequent patterns in this agreement process.
My dataset contains ~1000 disagreements(from ~20000 ratings). Each ratings is one of 6 categories. The categories are not ordered, but d, e and f represent different types of heart beats (incl. unknown), while a, b and c are other ECG patterns.
My initial idea was a parallel plot connecting each rating in for rater1, rater2 and common:

This gives the general idea, that there are som major patterns, but it is not easy to interpret.
Im am hoping someone can recommend a better solution.
Slightly modified sample of the data:
rater1,rater2,common
f,d,e
c,b,b
f,a,a
d,e,e
d,f,f
d,f,e
f,d,c
f,d,e
b,c,c
d,e,e
c,b,b
d,b,b
d,f,e
d,e,e
f,e,e
f,e,e
b,c,c
f,e,e
d,f,e
f,d,e
b,c,c
d,e,e
f,d,e
c,f,c
f,e,e
f,d,f
f,e,e
f,e,e
d,f,e
d,f,f
f,d,e
f,e,e
c,f,c
f,e,e
c,f,c
f,d,e
f,d,f
c,f,c
d,f,e
d,e,e
f,e,e
b,c,c
c,f,c
f,e,e
f,d,e
f,e,e
b,c,c
f,e,e
f,d,f
e,f,e

 A: This isn't as sexy as your plot, but it might make it easier to read off actual frequency data.

Simulated data -- color indicates the number of raters who initially agreed with the eventual common rating (i.e. 0, 1 or 2).
Code:
library(ggplot2)

theme_set(theme_bw())
theme_update(strip.background=element_rect(colour="white"))
theme_update(panel.border=element_blank())

## Simulated data with similar format
n <- 10^3
df <- data.frame(common=sample(letters[1:6], size=n, replace=T, prob=c(1, 2, 3, 4, 4, 4)),
                 stringsAsFactors=F)
df$rater1 <- ifelse(runif(n) < 0.5, df$common, sample(letters[1:6], size=n, replace=T))
df$rater2 <- ifelse(runif(n) < 0.5, df$common, sample(letters[1:6], size=n, replace=T))
for(var in c("common", "rater1", "rater2")) {
    df[, var] <- factor(df[, var], levels=letters[1:6])
}
df$rater1_label <- sprintf(ifelse(df$rater1 == "a", "rater1 = %s", "%s"), df$rater1)
df$rater1_label <- factor(df$rater1_label, levels=c("rater1 = a", letters[2:6]))
df$rater2_label <- sprintf(ifelse(df$rater2 == "a", "rater2 = %s", "%s"), df$rater2)
df$rater2_label <- factor(df$rater2_label, levels=c("rater2 = a", letters[2:6]))
df$agree_with_common <- as.character(1*(df$rater1 == df$common) + 1*(df$rater2 == df$common))
p <- (ggplot(df, aes(x=common, color=agree_with_common)) +
      scale_color_manual("", guide=F,
                         values=c("0"="#D55E00", "1"="#0072B2", "2"="#009E73")) +
      xlab("common rating") + ylab("count") +
      geom_histogram(fill="white") +
      facet_grid(rater1_label ~ rater2_label) +
      ggtitle("Histogram of common rating conditional on individual ratings"))
p
ggsave("ratings.png", p, width=10, height=8)

