# How to quantify whether the difference between two groups is relevant?

I'm sorry that I am not at all an expert in statistics, so my question may sound vague, or very easy.

I have a group of people that I divide into two categories, $A$ and $B$, depending on some visible characteristic (say blue-eyed vs brown-eyed). There are $n_A$ individuals in $A$ and $n_B$ individuals in $B$.

I am interested in a certain eye disease, so for a fixed period (1 year) I observe them, and in the end I count how many people of each group got the disease. Say I count $m_A$ in group $A$ and $m_B$ in group $B$. Those are not samples; I have counted them on the entire group.

What I want to determine is whether eye color is relevant to the outbreak of the disease, i.e., what I would naively call the correlation between eye color and disease outbreak. Hence: how can I quantify, as a function of $m_A$, $n_A$, $m_B$ and $n_B$, whether it is likely that eye color is correlated to getting the disease, or whether the fluctuations between both groups are just caused by chance?

From my naive understanding, one possibility would be to consider two hypotheses, one that says that the disease occurs with some probability $p$, independently over individuals, no matter the group; one that says that there is a probability $p_A$ in group $A$ and a probability $p_B$ in group $B$. One should then determine which of these hypotheses is more likely... but I'm not sure how to do that.

Edit: after some more investigation I'm starting to believe that the correct way to proceed would be perhaps to construct a contingency table containing $m_A$, $m_B$, $n_A - m_A$ and $n_B - m_B$, and use Fisher's test to compute the probability of observing data which is as imbalanced or more imbalanced (in favor of one eye color being positively correlated to the disease outbreak) if the null hypothesis (that disease outbreak does not depend on eye color) is true. Would this be a sensible way to proceed?

• Yes, the Fisher's Exact test approach would also be viable. Commented Jun 27, 2015 at 14:00

Relative risk would be a good candidate. It is a ratio of $p_A$ to $p_B$ and if the ratio is significantly different from 1 (aka, its confidence interval does not contain 1) then you can conclude that the risks are not the same in groups A and B. The formulas are not too difficult, and there are also online calculators out there. For instance, this site provides both.

If you want to get fancier by adjusting for other characteristics such as age, sex, or family history of the eye disease, then you'd need more sophisticated techniques. One of them is Poisson regression. It's not as simple as relative risk and I'd suggest you to bring the data and question to meet with a statistics specialist.

• Thanks for your answer! When you talk of $p_A$ and $p_B$, are you defining those as $m_A/n_A$ and $m_B/n_B$? If so, where do you account for the absolute values of $n_A$ and $n_B$ (i.e., following the intuition that larger populations allow more definite conclusions)? Would this be in the confidence interval? (If so I'm not sure how you do it.)
– a3nm
Commented Jun 27, 2015 at 13:21
• As for your second paragraph, I'm not interested in complicating the problem or adjusting for any other characteristics for now, I'm just trying to understand this simplified, abstract problem.
– a3nm
Commented Jun 27, 2015 at 13:24
• @a3nm, yes, $p_A$ is $m_A/n_A$. The values of the group sizes are included in the computation of the standard error as a reciprocal. If one of the groups is much smaller, the standard error will be higher, making the confidence interval wider and more likely to include the null value of 1. The relative risk itself does not convey group size information. Commented Jun 27, 2015 at 13:48
• OK, this seems to make sense to me, thanks a lot! Now I'm just puzzled by the relationship of this to contingency tables and Fisher's test, which I linked when editing my post. Are there any connection between the two formalisms?
– a3nm
Commented Jun 27, 2015 at 14:05