We have a population of n=1000, nine binary responses (0 symptom absent, 1 present), and 2 timepoints. Note that some of the responses are heavily skewed, only 1% of the subjects are in the "symptom present" category at baseline for the most skewed response.

We calculate the change over time as ratio: participants with symptoms present at time2 / participants with symptoms present at time1.

This leads to increases between 100% and 600%.

Now we want to examine the confidence intervals around these changes. How would you go about this?


In your case, you've calculated a special case of a Rate Ratio (RR) since you have $n=1000$ in your denominator at time $t=1$ and $t=2$. For independent data, you can calculate a confidence interval using a binomial model for risk using the following formula:

$RR = a / b$

$\log \left( RR \right) = \log(a) - \log(b)$

$\mbox{SE} \left( \log \left( RR \right) \right) = 1/a + 1/b$

So the 95% CI is

$\mbox{Lower} = \exp \left(\log(a) - \log(b) - 1.96 * (1/a + 1/b) \right)$

$\mbox{Upper} = \exp \left(\log(a) - \log(b) + 1.96 * (1/a + 1/b) \right)$

Note it's not a symmetric interval!

For dependent data, you can estimate a matched-pair odds ratio using a similar formulation. Except for these data you must consider the 2 by 2 contingency table of the following form

$$ \begin{array}{ccc} & \text{Status at Baseline} & \\ & Case & Control \\ \text{Status at Followup} & & \\ Case & a & b \\ Control & c & d\\ \end{array} $$

The Mcnemar's paired odds ratio is $b / c$.

Note this is actually an odds ratio, not a risk ratio. However, inference about whether the status at follow-up was more likely to produce cases is equivalently tested by determining whether this odds ratio is equal to 1.

The 95% confidence interval for the matched pair odds ratio is given by:

$\text{Lower} = \exp \left( \log \left( b / c \right) - 1.96 * (1/b + 1/c)\right)$

$\text{Upper} = \exp \left( \log \left( b / c \right) + 1.96 * (1/b + 1/c)\right)$

To actually get a confidence interval for the risk ratio, you would have to make some assumptions about the correlation structure of the data using a GLM. However, it's much more common to see analyses of paired binary data using the McNemar's test.

  • 1
    $\begingroup$ You have assumed independence between $a$ and $b$ which might not be true, since the description indicates a repeated measures design. $\endgroup$
    – Aniko
    Jan 13 '14 at 17:40
  • 1
    $\begingroup$ @Aniko I've supplied an answer that accounts for correlated data. Thanks for your suggestion it was a detail that I overlooked. $\endgroup$
    – AdamO
    Jan 13 '14 at 19:09
  • 1
    $\begingroup$ @Torvon please see my edit $\endgroup$
    – AdamO
    Jan 13 '14 at 19:29
  • 1
    $\begingroup$ Yes. I am suggesting you use an odds ratio instead of a relative risk. They are different quantities, though tests of hypotheses about them lead to similar inference about the association between an exposure and an outcome. For rare events, the odds ratio approximates the relative risk. However, it looks as though you have these quantities reversed, since the effect estimates are extremely large here. $\endgroup$
    – AdamO
    Jan 14 '14 at 16:37
  • 1
    $\begingroup$ Sorry, I think I confused odds with odds ratios. Odds ratios of 4 or larger are, generally, unbelievably large effect sizes! In general, we like to formulate tests of rates so that the rate is less than 0.5, or the odds are less than 1. However, if the effect is somehow more prevalent in the follow-up period, it makes sense that the odds ratios would be so large. It would be good to know the prevalence of these events. $\endgroup$
    – AdamO
    Jan 18 '14 at 21:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.