1
$\begingroup$

A clinical trial is being planned, in which the primary outcome is binary (success/failure). There are two treatment groups (treatment/control). The aim of the trial is to show that the treatment is better. The outcome is bad, meaning that in the treatment group, we wish to have a smaller proportion.

The events of interest are rare, and assumed to be around 7% in the control group and 5% in the treatment group.

I did a lot of reading, and some people say that with low rate of events it's better to use the risk ratio (relative risk) over the proportion difference (with normal approximation). It's not clear to me why.

I wanted to ask, under the conditions described above, what is best: Fisher's Exact Test, Odds Ratio (with a CI or hypothesis testing of OR=1), or Relative Risk (with CI or hypothesis testing of RR=1)? Will I get similar results in all? Should sample size be roughly the same?

Thank you !

|cite|improve this question
$\endgroup$
2
$\begingroup$

Such guidelines exist, but either the odds ratio (OR) or the relative risk (RR) is fine. You only have to be sure that you interpret either one correctly. This is the advice given by Schmidt & Kohlmann (See below). Given that the relative risk is more intuitive, then it may be more advisable to use it. At least, this is the advice given by Deeks (See below). Deeks recommends the odds ratio only when it is obtained from case-control studies and logistic regression analysis.

Additionally, the CI would be much better at communicating the uncertainty about the effect than using a p-value. Both are analogous either way.

An additional outcome of interest for you could be the Number Needed to Treat, which is calculated from the absolute risk. In your situation using the 5% and 7% values you posted, one would need to treat ~50 people to reduce incidence by 1 person.

This tool calculates all 4 metrics given the 2x2 contingency table, and returns confidence intervals using a variety of methods. It makes use of the epitools R package. I'd be glad to answer any questions.

Schmidt, C. O., & Kohlmann, T. (2008). When to use the odds ratio or the relative risk? International Journal of Public Health, 53(3), 165–167. https://doi.org/10.1007/s00038-008-7068-3

Deeks, J. (1998). When can odds ratios mislead? Odds ratios should be used only in case-control studies and logistic regression analyses. BMJ (Clinical Research Ed.), 317(7166), 1155-6-7. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/9784470

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Firstly, thank you for the wise comment. I have two questions. 1. Should the conclusion of all methods be identical (i.e., if one test shows a difference between groups, so will the others). 2. Should the required sample size for all methods be roughly identical ? i.e., is using OR instead of Fisher's exact test reduces the sample size significantly? I am trying to figure out why people recommended using the OR and RR when the proportions are low. $\endgroup$ – user3275222 Jun 17 '17 at 18:42
  • $\begingroup$ No, the methods are not identical as the different methods for calculating the confidence intervals can lead to different inference. The point estimates will have the same direction though. Unless it is a theoretical question, the solution when one comes up with issues like this is to perform all the valid options, then attempt to understand any reasons for differences in results if there are. If there are not, then it increases confidence. For your second question, I do not understand how a test reduces sample size. And can you provide a reference for that recommendation? $\endgroup$ – Heteroskedastic Jim Jun 17 '17 at 18:57
  • $\begingroup$ I ran a calculation using a sample size software, and for an OR test I got a much smaller sample size than for the exact test. I think it has something to do with the fact that the OR test or CI uses a normal approximation ? $\endgroup$ – user3275222 Jun 17 '17 at 19:34
  • $\begingroup$ Are you talking about a "power" analysis? $\endgroup$ – Heteroskedastic Jim Jun 17 '17 at 19:35
  • $\begingroup$ Yes ! Also about the analysis, but thinking about both aspects. I just ran a simulation in SAS, and the power came out identical to OR (95% CI) and Fisher. $\endgroup$ – user3275222 Jun 17 '17 at 19:49
0
$\begingroup$

It may depend on the sample size but keep in mind that Fisher's Exact test is very conservative is based on the assumption that both sets of marginals are fixed (number in each group and number of each type of outcome). Logistic regression and Pearson Chi Square are both possibilities. If you have a browser that works with unsigned applets, you can investigate the performance of the Chi Square test with different sample sizes here.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Hi David. Since the effect is very small, the sample size is expected to be very large, hundreds if not thousands of subjects. $\endgroup$ – user3275222 Jun 17 '17 at 18:24
  • $\begingroup$ I would say Pearson Chi Square and logistic regression are equally good choices. Choose one a priori and stick with it. Don't try both. $\endgroup$ – David Lane Jun 17 '17 at 18:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.