1
$\begingroup$

I'm following this example (see below for the source):

Breast Cancer No Breast Cancer Total
DDT exposed 360 13636 13996
Unexposed 1079 76313 77392
RR = 1.87
CI = 1.66 - 2.10

What I do not like is this: if I multiply the number of "No Breast Cancer" by a large number, even one million or more, these two numbers do not change at all. But now the occurrence of the event is very rare and the difference from 360 to 1079 may well be due to random chance.

I'm surprised that the ratio between the positive samples and the whole population is not even considered when we compute the confidence interval.

Intuitively 360 out of 13636 is different from 360 out of 136,360,000 and I would expect the confidence in the measurement to reflect this by being much lower.

Am I missing something? Thanks.

EDIT: I'll try to explain it with an example:

First experiment. My sample is 1000, half of this is young, half is old. I do a random split, the worst case is to have a "shift" of 250 units from the average. Now I run my tests and I get 5 cases in group A and 25 in group B.

Second experiment, completely different and independent from the previous one. My sample is one million, half young, half old. I do a random split. I think the average "shift" I can expect from the split is very high, I think it is possible to compute it, probably higher than 250. Now I run my tests and I get 5 cases in group A and 25 in group B, just like before.

Isn't the "signal" in the second experiment too weak to conclude something with the same "confidence" of the first one?

Source:

Please note that the numbers are slightly different from the ones in the table in the original example but match the one used in the formula.

"Confidence Intervals for the Risk Ratio (Relative Risk)"

"A Nested Case-Control Study"

$\endgroup$
4
  • $\begingroup$ Re "The difference from 360 to 1079 may well be due to random chance." This is incorrect: perhaps that misconception is the basis for your question? $\endgroup$
    – whuber
    Commented May 24, 2021 at 15:21
  • $\begingroup$ @whuber Why? Let's say I'm doing a trial splitting my population randomly in half. If the population is very large I would expect to have a large absolute source of error or "noise" from the randomness of the split itself, that might even be bigger than the effect I'm trying to measure. If the population is small, the probabilty of an "unbalance" of the same magnitude is smaller if not impossibile. $\endgroup$
    – lorenzo
    Commented May 24, 2021 at 16:45
  • $\begingroup$ That's a common intuition -- but it's incorrect. The sampling uncertainty is essentially unrelated to the population size, unless the population is less than about ten times the sample. I provide some guidance at stats.stackexchange.com/a/31571/919. $\endgroup$
    – whuber
    Commented May 24, 2021 at 16:51
  • $\begingroup$ @whuber Sorry, the post did not help me. I updated my question with an example. $\endgroup$
    – lorenzo
    Commented May 24, 2021 at 19:01

1 Answer 1

2
$\begingroup$

The "relative risk" of 1.86 is actually an odds ratio, calculated by $A \times D / (B \times C) =$360 * 76313 / (13636 * 1079). Notably, entries B and D are not a denominator, they are non-cases, so if you multiply non-cases by some constant, your OR remains unchanged but the RR will change.

Now your confusion seems to come from the idea that you've been told that the odds ratio approximates the relative risk when the outcome is "rare". So you are asking, what happens when, instead of tens of cases, you have hundreds or thousands of cases. Isn't the outcome no longer "rare"?

The answer is that this "rareness" has to do with the prevalence of cases in the population or sampling frame rather than the overall number of cases in the sample. Suppose for instance, I perform a case control study of children diagnosed with Hunter's syndrome among a hospital network in a small region of the US to model the association with family history as a motivator to perform a family-based study. There might be 10 such kids over a one year retrospective review, and an odds ratio of 5.4 with any family history. Then I partner with a clinical trial network and I identify over 1,000 such kids with over 1,000 well-matched controls. The odds ratio is still 5.4, and it still approximates the risk ratio, to provide strong evidence of the heritability of the condition.

The variance of the CI will change in these conditions. For the log odds ratio, the variance is given by a nifty formula that you should memorize

$$ \text{var}(\log (OR)) = 1/A + 1/B + 1/C + 1/D$$

where $A, B, C, D$ are the cells in the 2 by 2 tabular array of outcomes. In other words, recruiting more patients will increase the precision of the estimates, leading to narrower CIs, and better power.

$\endgroup$
3
  • $\begingroup$ thanks for the long answer. In your conclusion you say that recruiting more persons improves the CI because you assume that this will also increase the occurrences and I agree. Instead I want to compare the "strength/quality" of two different experiments one with a small sample and one with a large one but with the same number of positives. The larger experiment seems weaker to me. I've updated my question with one example. If this is true, I expected to be able to see this from the CI but it is not the case. $\endgroup$
    – lorenzo
    Commented May 24, 2021 at 19:23
  • 1
    $\begingroup$ @lorenzo your two experiments will not produce the same odds ratios because you're increasing the "non-cases" by a non-multiplicative amount. They'll be almost the same. You'll see some improvement by sampling 1,000,000 instead of 1,000. But actually write out the variance expression you have V=1/5 + 1/25 + 1/(big number) + 1/(big number). Make big number bigger won't give you THAT much more precision. $\endgroup$
    – AdamO
    Commented May 24, 2021 at 19:29
  • $\begingroup$ if I multiply the "No Breast Cancer" column by 10000 I see no changes in RR and an extremely small difference in CI. But I consider the results of this new case to be much "weaker" than the original one, do you agree?. The positives are so small, 0.0003%, that just about anything could sway the result in real life. Variance is going to be smaller. $\endgroup$
    – lorenzo
    Commented May 24, 2021 at 20:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.