Summarizing ratios in a systematic review - does any joint effect measure exist?

Question

I am responding to reviewer comments on a systematic review summarizing a number of risk factors. The exposure is any risk factor (this is what I am trying to make an overview of) and the outcome is first event of physical child abuse. The risk factors have been reported using various effect measures: relative risks (RR), odds ratios (OR), incidence rate ratios (IRR), and hazard ratios (HR). The study type varies, including prospective studies and cross-sectional designs.

I have a reviewer commenting that it is difficult to compare across these different effect measures, to which I agree - but am I correct to say that there is no single effect measure to which the others can be translated, that accurately would reflect the original estimate without using unreasonable assumptions?

For almost all papers the ratio estimates are adjusted for a number of covariates, so I can't just calculate them without access to the raw data. For some of these (maybe all?), the values would approach each other if the event studied is sufficiently rare - but I cannot guarantee it will be so in all studied populations.

I have tried to find an answer in Cochrane's handbook for systematic reviews, and looked around for primary articles with good solutions, but to no avail. This review (sorry, it is not open access): https://doi.org/10.1016/j.avb.2006.03.006 seems to join various effect measures in r-, d- and g-values, but using a program that has since been abandoned, and since I haven't seen this mentioned in Cochrane, I get the feeling that this may not be the desired approach anymore. It seems the calculations mentioned there is for standardized mean differences, and this would also only be appropriate for a continuous outcome - mine is dichotomous. I have also been through a number of systematic reviews on risk factors from various fields, either finding that these articles just postulates that everything is in odds ratios, report various measures as I do or simplify things, for example only reporting whether something is a risk or not, but no measure of magnitude.

Any advice would be highly appreciated.

Answer 1

I have consulted some sources, including useful comments from The_old_man and local academics here, and there are several answers, simplest is first:

Overall, to the best of my knowledge, there is no such thing as a joint measure summarizing Odds Ratios (OR), Incidence Rate Ratios (IRR), Risk Ratios (RR), and Hazard Ratios (HR).

Some conversions are possible - OR can be converted to RR and vice versa if the prevalence of the outcome in the control group is known. This is detailed in this blog (thanks for the question, user Amorphia): https://www.r-bloggers.com/2014/01/how-to-convert-odds-ratios-to-relative-risks/ However, be aware that relative risk only makes sense in a prospective study - for example it would be numerically possible, but nonsense, to calculate relative risk in a cross-sectional study. Risk needs to take place during some time span. Odds ratio, however, can be used in a variety of study types.

Also, HRs can be approximated using various techniques. HRs are currently recommended by Cochrane as the best measure for time-to-event data. The techniques for estimates are detailed here: training.cochrane.org/handbook/current/ section 6.8 in the current version. However, be aware that this makes a lot of sense for randomized studies that are already "controlled" for covariates, but not so much in observational studies, as these techniques does not take confounding into consideration. Also, for causal purposes, be aware of the shortcomings of HR in comparison to RR, see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3653612/

In comparisons, HRs and IRRs can be more or less thought of as the same thing - this is backed up by this article: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3653612/

If the event in question is sufficiently rare, OR, RR, IRR, and HR all approximate each other.

However, none of the conversions or comparisons can be done without either further information, or assumptions that may or may not hold. If the measure of interest is risk, the following relationship holds (Modern Epidemiology, Third Edition):

$RR < (IRR \approx HR) < OR$

And this can be used to compare the measurements as is (but of course will not allow meta-analysis).

A final option is simply to state the direction of risk (for example +: risk goes up, -: risk goes down, 0: not associated) and abandon the magnitude altogether - this holds for all measurements mentioned.

Answer 2

In my opinion:
a) When both the baseline risk and the estimate size are small (eg <0.1 and <2.5 respectively), no conversion is needed for meta-analysis; since RR, HR, and OR lie very close (1).
b) When baseline risk is high (eg >0.2 (2)), there are good methods for converting HR -> RR, and OR -> RR; which yield a bias of 25% at most in ideal cases (3).

If anyone has been able to find an optimal conversion for HR <-> OR please share it with use

Ref

1. Symons MJ, Moore DT. Hazard rate ratio and prospective epidemiological studies. J Clin Epidemiol. 2002 Sep;55(9):893–9.
2. VanderWeele TJ. On a Square-Root Transformation of the Odds Ratio for a Common Outcome. Epidemiology. 2017 Nov;28(6):e58-e60. doi: 10.1097/EDE.0000000000000733. PMID: 28816709; PMCID: PMC5617805.
3. VanderWeele TJ. Optimal approximate conversions of odds ratios and hazard ratios to risk ratios. Biometrics. 2020 Sep;76(3):746-752. doi: 10.1111/biom.13197. Epub 2020 Jan 6. PMID: 31808145.