2
$\begingroup$

I would appreciate if anyone can share a reference that discusses magnitudes of hazard ratios as effect sizes. Specific subject matter obviously weighs in when trying to determine what a relevant effect size is, but the problem I have seen in some recent papers is that time-to-event Cox PH analyses with large sample sizes (i.e., in the thousands, that's a large sample in health sciences) result in some arguably tiny hazard ratios labeled 'significant' and the authors take that as evidence of relevance or consequence. The large sample size fallacy is nothing new, but it's difficult to argue about magnitude of an effect when there is no general frame of reference. For instance, I am aware that an odds ratio can be converted into a standardized mean difference (in the log odds scale) as shown in this paper http://www.ncbi.nlm.nih.gov/pubmed/11113947 , for which, thanks to Cohen (1988), there are some generally agreed magnitudes for 'small', 'medium' and 'large' (at least in the social and behavioral sciences). Not that these magnitude guidelines are set in stone, but they do have some justification, and Cohen explains they are just general qualitative definitions.

Cohen J. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd Ed. Hillsdale, NJ: Laurence Erlbaum Associates

$\endgroup$

4 Answers 4

8
$\begingroup$

Cohen's approach been increasingly criticized. In general, attempting to define cutoffs for "small", "large", etc. is futile. The interpretation of any statistical measure is context-dependent.

What is useful is to supplement a relative effect (hazard ratio, odds ratio, etc.) with an absolute effect. Suppose one had a model with only age and sex as predictors. An absolute effect estimate might be the difference in estimated 5-year survival probabilities for males at age x with that of females at age x, where x is some convenient value such as the median age in the combined samples.

Update: Here is a graph connecting hazard ratios to absolute risk changes.

$\endgroup$
1
  • 1
    $\begingroup$ (+1). Even Cohen criticized his own "small/medium/large" guidelines from the beginning, and clearly and repeatedly advised against using them! He thought that these thresholds might have some value but "only when no better basis for estimating the ES index is available", in his own words. This was probably a mistake from his part to establish these standards, given that his warnings seem to be routinely ignored. Also, I wonder in what kind of situation it is impossible to use one's own judgement to decide how strong or interesting an effect size is (but maybe that's ignorance from my part). $\endgroup$
    – J-J-J
    Commented Sep 9, 2023 at 7:21
4
$\begingroup$

I don't think there is, and I think that there should be no guideline for such interpretations. A hazard ratio can be judged as large or small depending, for example, on the scale of the covariates (this is why it is good practice to standardize such variables).

In a power analysis for two groups, for example, the effect depends on the amount of events in the two groups, the total sample size, etc. For different data sets, "large" might be completely different values; you can play around with a tool like this.

Finally, in the case of two groups, assuming proportional hazards, a hazard ratio of 5 means that an individual in the high-risk group is 5 times more likely to have an event as compared to an almost identical individual in the low-risk group. Subjectively, any hazard ratio above 3 would definitely catch my eye as being large, however with some reservations.

$\endgroup$
2
$\begingroup$

This reminds me of a similar grey area. What level of significance are you willing to accept: 0.01 or 0.05. As someone explained to me, if the research is about how to keep a plane from falling out of the sky then we will be looking for at least a 0.01 probability that our results are not due to chance. My point is, it depends on the question being researched - o.1 effect size could be described as 'very important' rather than 'small' if it is the first indication that research in a given area has potential. Similarly (in public health research, as an example) if the cost of obtaining this 'small' effect size is minimal, 0.1 could be hugely important - calling it small is misleading.

$\endgroup$
0
$\begingroup$

I too am interested in this question. For what it's worth, I found one article that attempts to define "small, medium, large" hazard ratios for binary or continuous covariates. However, the article is not very well cited (only 13 google scholar citations as of 3/24/19):

Azuero, Andres. "A note on the magnitude of hazard ratios." Cancer 122.8 (2016): 1298-1299.

$\endgroup$
1
  • 1
    $\begingroup$ That note is specific to a distributional assumption for the survival time (exponential distribution), so it is of limited applicability. $\endgroup$
    – user16263
    Commented Apr 3, 2019 at 18:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.