Combining multiple contingency tables I have images that are evaluated by different experts. Basically, each expert look at the same pictures and saying whether a feature in the image exists or not. We have some images from diseased and normal people. The idea is to find whether a particular feature of the image is associated with disease. IF there were only a single expert we would be able to do a Fisher’s exact test. But there are multiple experts so we need a way to combine multiple contingency tables to get a single p value. By digging internet, I found that “Cochran-Mantel-Haenszel statistics” does something similar to this. However, I still have questions whether this is the appropriate test. Any suggestions?
 A: I think you can simplify this if you stop thinking of it as a collection of contingency tables, and re-envisage it as a simple linear model (with binomial response ie logistic regression).  


*

*The response variable is whether the image is actually from someone with a disease (I presume you have this information, and also that the experts did not know this in advance, as otherwise the whole exercise seems a bit pointless).

*The explanatory variable is how many experts (or perhaps what proportion of experts) rated the image as having the feature you're interested in.  If there are actually multiple features, you can have as many of these explanatory variables (and indeed interactions) as you want.


Then you can just fit this as a logistic regression model, or possibly as a generalized additive model with a binomial response if you want to allow for non-linearity between the "proportion of experts who saw the feature" and the response.
Alternatively (but same basic idea), you could use a similar approach but make the "number of experts who saw the feature" an ordered categorical explanatory variable.  This would make sense if you think (for example) there is a radical discontinuity between 0, 1 or 2 experts who see the feature rather than a smooth progression of "the more who see it, the more likely it is to be there.".
