How to evaluate concordance between 2 raters and factors that affect the concordance I have about 1000 subjects ($id$) and each of them has some baseline variables (demographic variables $X1, X2, X3, Xn$, which can be continuous, factors). Every subject is classified by two raters ($rater1$, $rater2$) in 2 factors. 


*

*How can I evaluate and describe the concordance between the raters?

*How can I evaluate which, if and how the demographic variables $X1, X2, X3, Xn$ may affect this concordance. 

*Within each class of $rater1$ or $rater2$, how can I evaluate the extend to which the variables $X1, X2, X3, Xn$ resemble each other (the subjects have similar $X1, X2, X3, Xn$ variables)


Any help or information material would be of great help
 A: *

*There are several procedures and measures that you can caluclate to describe the so-called inter-rater agreement. One of the most famous is Cohen's kappa. You can calculate it in R with the function kappa2 from the irr package. Another measure is the intraclass-correlation. You can calculate it in R with the icc function from the irr package. Here is a good intro to concordance analysis. For a visual comparison of two raters, Bland-Altman plots are often used. The R package ResearchMethods (link) provides an implementation of the Bland-Altman plot (the function BlandAltman).

*One way I can think of doing it would be to create a binary variable which is 1 if the ratings are concordant and 0 if the ratings are discordant between the raters. Then, you could run a logistic regression with this new binary variable as dependent variable and the demographic variables as independent variables. This might provide some insight whether the concordance depends on demographic variables. The exponentiated coefficients are the odds ratios: $OR=e^{\beta}$. An odds ratios greater than 1 means that for one unit increase in a continuous demographic variable (like age), the odds for a concordant rating increase whereas the opposite istrue for odds ratios < 1. If the demographic variable is categorical, the odds ratios are usually calculated with respect to a reference group, for example age 30-40 vs. 20-30. If the odds ratio for the persons aged 30-40 is > 1, this would mean that a concordant rating is more likely for people aged 30-40 years compared to persons aged 20-30 years. Here is a nice explanation of how to interpret the output of a logistic regression. 
Or maybe you could run two separate multivariate regressions for each rater independently with the rating as dependent and the demographic variables as independent variables. In this way you might identify differences in the importance of the demographic variables on the ratings of each rater: For rater1, age might play an important role in building the ratings whereas in rater2 age might be unimportant.
