# More interpretable measure of association than odds ratios for contingency tables with 0 counts

How strong/weak of a correlation is this?

                  Exposure
No Yes  Total
FALSE 139 467   606
Disease     TRUE   11 104   115
Total 150 571   721

OR =  2.81
Exact 95% CI =  1.45, 5.97
Chi-squared = 10.49, 1 d.f., P value = 0.001
Fisher's exact test (2-sided) P value = 0.001


                   Exposure
No  Yes  Total
Disease     FALSE  301 1232  1533
TRUE     0   17    17
Total  301 1249  1550

OR =  Inf
Exact 95% CI =  1, Inf
Chi-squared = 4.14, 1 d.f., P value = 0.042
Fisher's exact test (2-sided) P value = 0.057

• Are there any better/other methods for calculating relative risk with a confidence interval for data like these?

So, the first one is pretty easy, we could say "we estimate that those with the exposure are 2.81 [1.45, 5.97] times more likely to have the disease (p=.001)". That sounds like a pretty strong finding. Surprisingly strong for the hypothesis I'm working with.

The second table doesn't work with the same formula, "we estimate that those with the exposure are infinitely more likely to have the disease [1, Inf], p < .05". What can be said about the second table? Shrug and say, "hrm, look at that, we are missing about 4 not exposed with the disease .. and there are none who have the disease who weren't exposed." I guess I already know the answers here. But, on the second table, I'm torn between thinking, "OMG, LOOK, UNEXPOSED HAVE 0, and p<.05!" and thinking, "that's interesting that that happened completely by chance". I guess that's when we say, "more research needed"?

-
What part of interpretting the strength of the relationship are you struggling with? One piece of advice I can give is that the second table will struggle to find any relationship because of the low count of disease=TRUE. The p-value is mainly telling you that Pr(disease=False & exposure = yes) is well predicted by the marginal counts. – probabilityislogic Aug 20 '12 at 7:46
is there a better test for this more rare disease? – Skylar Saveland Aug 20 '12 at 14:21
In epidemiology, you may add a small number like 0.5 to all four cells to get an estimate of the odds ratio. Just remember to state that in the footnote or description. However, diseases showing up only in the exposed group itself is an important finding. It's better not to get so hung up on making an odds ratio, and just report what the rates for exposed and unexposed separately. – Penguin_Knight Aug 21 '12 at 1:52

I am not exactly sure what you want to get finally, but have a look at this mosaic plots, testing independence:

And for the second dataset:

In both cases the data is dependent, but it is dependent in different manner here: if about the first plot we can just tell that 11 is too small (comparing to the whole table), then about the second plot we can tell that 0 is too small comparing to 17.

P.S. I have cheated here a little to create the second plot, I changed the data to get rid of zero:

3010, 12320

1, 170

UPD 1. R code for the mosaic plots:

x <- matrix(0, ncol=2, nrow=2)
x[1,] <- c(139, 467)
x[2,] <- c(11,  104)


UPD 2. Few words about what this plot shows. The area of each cell is proportional to the number of samples with the combination of the properties. That is just an easy way to visualize such tables as provided in the question.

The more interesting thing is the colors ("shade" parameter). It performs something like Chi Squared test: compares the theoretical distribution when the independence hypothesis is true with the given distribution. Then the large deviations are colored: the more significant the deviation, the more saturated the color.

Unfortunately this tool only checks the dependencies in data, but that is usually all you need in case of two variables and four possible observation.

-
So, we can't say much more than that the data is dependent? Are there other, better methods of estimating relative risk/odds or is that about as good as I can do? – Skylar Saveland Aug 20 '12 at 23:12
These are nice, why don't you post the code, perhaps a link to some info about mosaic plots, & a couple of sentences of accompanying explication? – gung Aug 21 '12 at 0:09
@SkylarSaveland One can conclude from what we achieved that exposure is more significant in second case, but I am not sure we can conclude that from such little data. And unfortunately I don't know about any other techniques to analyse this data :( – Dmitry Laptev Aug 21 '12 at 10:03
@gung I have updated the answer to provide code example, it is extremely easy I must say... – Dmitry Laptev Aug 21 '12 at 10:04
Thanks, @DmitryLaptev. I didn't think it was difficult, though. My idea's just that the best answers, & the ones that serve as a repository of accessible & actionable knowledge, are fairly comprehensive. Your answer here is good, but requires some reading between the lines; in addition to the code (& you could have done just 1 as a demo), it might be nice if you linked to a source of more info on mosaic plots (such as this), & maybe a sentence or two explaining explicitly why / how they are better than ORs, & their pros & cons. – gung Aug 21 '12 at 14:03

A convenient parameterisation of this problem is through the marginal and conditional probabilities. So we have a parameter for exposure $\pi_{E}$ and two parameters for disease given the exposure: $\pi_{D|E}$ and $\pi_{D|\overline{E}}$. Then we do a hypothesis test

$$H_0:\; \pi_{D|E}=\pi_{D|\overline{E}}$$

I would have thought that a good measure of association is $1-P(H_0|nI)$ where $n$ is the sampled counts and $I$ is the prior information/assumptions.

Alternatively, you could calculate the posterior distribution of the difference $\delta=\pi_{D|E}-\pi_{D|\overline{E}}$ or the odds ratio $\gamma=\frac{\pi_{D|E}}{\pi_{D|\overline{E}}}$ if you prefer. One thing to note is that because the disease count is small, you inference will be sensitive to the prior distribution - so a sensitivity check for a few similar priors would be desirable.

There is also a fair amount of literature on Bayesian approaches. here is a useful blog entry which discusses an example of what I'm talking about.

-

In my mind, as an Epidemiologist, it depends on why there were zero counts.

If there is a particular combination of exposure and disease that is known to be possible but rare, then the usual way to proceed is to add some small number to each cell, usually 0.5 or so, and proceed from there, usually using Exact statistics to produce an OR that is interpretable in the usual fashion.

But its possible that there's a reason that there's a 0 cell count there. It might be etiologic, at which point that's its own major finding, or it might be revealing a flaw in the study design - for some reason you aren't sampling that particular combination, they're being excluded from the study, etc.

Both warrant further investigation, and in both cases you've moved beyond a single number emerging from a contingency table.

-