Correct approach for computing interrater reliability Can't find a strong answer to this "special case" elsewhere on the interwebs, so I'm hoping you can help. 
I want to computer interrater agreement (actually, in this case, inter-method agreement) for the following scenario: The same person provides nominal ratings of whether or not an event occurred each day for 30 days, and I have 10 people doing this.  What's the appropriate statistical approach for calculating overall agreement (e.g., kappa, or something else)?  Data looks like this (but I can reshape, if needed): 

 A: It's not clear what ev1 and ev2 are in your example or how this constitutes inter-method reliability. I'm also not sure what you are trying to calculate the reliability of.
Let's assume you have each person provide ratings each day using two different methods and you want to calculate the reliability of these two methods. In that case, you can create a separate 30x2 matrix for each participant where each row corresponds to a day and each column corresponds to a method. You can then calculate any index of nominal agreement or reliability on these matrices and provide an average.
If, on the other hand, you have each person provide rating each day using a single method and you want to calculate the reliability of all raters. In this case, you can create a single 30x10 matrix where each row corresponds to a day and each column corresponds to a rater. You can then calculate any index of nominal agreement that allows for more than 2 raters.
Finally, if you have two methods and want to estimate the reliability of both methods and raters at the same time, you could perform a generalizability study using methods and raters as facets. 
