3
$\begingroup$

I am responding to reviewer comments on a systematic review summarizing a number of risk factors. The exposure is any risk factor (this is what I am trying to make an overview of) and the outcome is first event of physical child abuse. The risk factors have been reported using various effect measures: relative risks (RR), odds ratios (OR), incidence rate ratios (IRR), and hazard ratios (HR). The study type varies, including prospective studies and cross-sectional designs.

I have a reviewer commenting that it is difficult to compare across these different effect measures, to which I agree - but am I correct to say that there is no single effect measure to which the others can be translated, that accurately would reflect the original estimate without using unreasonable assumptions?

For almost all papers the ratio estimates are adjusted for a number of covariates, so I can't just calculate them without access to the raw data. For some of these (maybe all?), the values would approach each other if the event studied is sufficiently rare - but I cannot guarantee it will be so in all studied populations.

I have tried to find an answer in Cochrane's handbook for systematic reviews, and looked around for primary articles with good solutions, but to no avail. This review (sorry, it is not open access): https://doi.org/10.1016/j.avb.2006.03.006 seems to join various effect measures in r-, d- and g-values, but using a program that has since been abandoned, and since I haven't seen this mentioned in Cochrane, I get the feeling that this may not be the desired approach anymore. It seems the calculations mentioned there is for standardized mean differences, and this would also only be appropriate for a continuous outcome - mine is dichotomous. I have also been through a number of systematic reviews on risk factors from various fields, either finding that these articles just postulates that everything is in odds ratios, report various measures as I do or simplify things, for example only reporting whether something is a risk or not, but no measure of magnitude.

Any advice would be highly appreciated.

$\endgroup$
8
  • 1
    $\begingroup$ Could you please explain your analysis in more detail? What is the outcome, how is it measured, etc? You have mentioned hazard ratio which is the default effect measure for analyzing time-to-event studies. In contrast, odds ratio and relative risks are used for dichotomous outcomes, and can be converted to each other. So I am a bit confused by your study analyses here. $\endgroup$ Jan 16, 2023 at 12:19
  • $\begingroup$ @The_old_man thank you for your attention! I have tried to edit accordingly. The outcome is dichotomous, and analyzed using various methods across the articles, including both OR, RR, IRR, and HR. How would you convert an adjusted RR to OR? Or vice versa? $\endgroup$
    – troelsgk
    Jan 16, 2023 at 12:30
  • 1
    $\begingroup$ In general, risk=odds/(1+odds) and odds = risk/(1-risk). For converting ratios, you need prevalence in the reference group. You can easily find websites that calculate this. Now, regarding your main question, what you described is a time-to-event study and the most appropriate way of summarizing time-to-event data is to use methods of survival analysis and express the intervention effect as a hazard ratio. I recommend you check the Cochranes handbook chapter 6, section 6.8.2: training.cochrane.org/handbook/current/chapter-06 $\endgroup$ Jan 16, 2023 at 12:44
  • $\begingroup$ Dear @The_old_man That chapter is the one I mention in the question - is there any methods there you think about in particular? I also stumbled upon the method with prevalence in the reference group. I am however in doubt how I would use that in real-life calculations? With a prevalence in the article it is straight-forward, but what if not? Would you assume a generic one and apply it across all conversions? In this review, I do not have access to the raw data, so cannot run time-to-event analyses. I hoped for something like Cohen's D, but for non-continous outcomes, but this may not exist? $\endgroup$
    – troelsgk
    Jan 16, 2023 at 13:12
  • 1
    $\begingroup$ If you don't have the prevalence, you really can't convert. However, if the overall prevalence of the event is below 10%, OR and RR are quite close in value. Basically in 6.82, Cochrane describes 3 ways for estimating HRs. If you have these information in your included studies then you can calculate HR which is the ideal scenario. However, as is usually the case, you won't have these data. Check out this answer and read comments carefully:stats.stackexchange.com/questions/130237/… $\endgroup$ Jan 16, 2023 at 13:28

1 Answer 1

2
$\begingroup$

I have consulted some sources, including useful comments from The_old_man and local academics here, and there are several answers, simplest is first:

Overall, to the best of my knowledge, there is no such thing as a joint measure summarizing Odds Ratios (OR), Incidence Rate Ratios (IRR), Risk Ratios (RR), and Hazard Ratios (HR).

Some conversions are possible - OR can be converted to RR and vice versa if the prevalence of the outcome in the control group is known. This is detailed in this blog (thanks for the question, user Amorphia): https://www.r-bloggers.com/2014/01/how-to-convert-odds-ratios-to-relative-risks/ However, be aware that relative risk only makes sense in a prospective study - for example it would be numerically possible, but nonsense, to calculate relative risk in a cross-sectional study. Risk needs to take place during some time span. Odds ratio, however, can be used in a variety of study types.

Also, HRs can be approximated using various techniques. HRs are currently recommended by Cochrane as the best measure for time-to-event data. The techniques for estimates are detailed here: training.cochrane.org/handbook/current/ section 6.8 in the current version. However, be aware that this makes a lot of sense for randomized studies that are already "controlled" for covariates, but not so much in observational studies, as these techniques does not take confounding into consideration. Also, for causal purposes, be aware of the shortcomings of HR in comparison to RR, see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3653612/

In comparisons, HRs and IRRs can be more or less thought of as the same thing - this is backed up by this article: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3653612/

If the event in question is sufficiently rare, OR, RR, IRR, and HR all approximate each other.

However, none of the conversions or comparisons can be done without either further information, or assumptions that may or may not hold. If the measure of interest is risk, the following relationship holds (Modern Epidemiology, Third Edition):

$RR < (IRR \approx HR) < OR$

And this can be used to compare the measurements as is (but of course will not allow meta-analysis).

A final option is simply to state the direction of risk (for example +: risk goes up, -: risk goes down, 0: not associated) and abandon the magnitude altogether - this holds for all measurements mentioned.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.