17
$\begingroup$

One way to summarize the comparison of two survival curves is to compute the hazard ratio (HR). There are (at least) two methods to compute this value.

  • Logrank method. As part of the Kaplan-Meier calculations, compute the number of observed events (deaths, usually) in each group ($Oa$, and $Ob$), and the number of expected events assuming a null hypothesis of no difference in survival ($Ea$ and $Eb$). The hazard ratio then is: $$ HR= \frac{(Oa/Ea)}{(Ob/Eb)} $$
  • Mantel-Haenszel method. First compute V, which is the sum of the hypergeometric variances at each time point. Then compute the hazard ratio as: $$ HR= \exp\left(\frac{(Oa-Ea)}{V}\right) $$ I got both these equations from chapter 3 of Machin, Cheung and Parmar, Survival Analysis. That book states that the two methods usually give very similar methods, and indeed that is the case with the example in the book.

Someone sent me an example where the two methods differ by a factor of three. In this particular example, it is obvious that the logrank estimate is sensible, and the Mantel-Haenszel estimate is far off. My question is if anyone has any general advice for when it is best to choose the logrank estimate of the hazard ratio, and when it is best to choose the Mantel-Haenszel estimate? Does it have to do with sample size? Number of ties? Ratio of sample sizes?

$\endgroup$
  • $\begingroup$ How do these estimates relate to the one given by Cox regression? That's gotta be the gold standard for estimating HR. $\endgroup$ – Aniko Aug 18 '10 at 16:41
  • $\begingroup$ Cox model incorporates covariates. The Kaplan-Meier, Nelson-Aalen, Mantel-Haenszel methods model hazard as a function only of age. $\endgroup$ – shabbychef Aug 19 '10 at 18:54
  • $\begingroup$ @shabbychef: with Cox PH, use a single binary covariate, i.e. coded 0/1 for reference/comparison groups, then exp(beta) = HR. $\endgroup$ – ars Aug 20 '10 at 6:47
  • $\begingroup$ The log-rank is a more powerful test than Cox PH when the proportionbal Hazards assumption is satisfied. So with a single 2-level covariate, a log-rank or Mantel-Haenszel test is preferable. $\endgroup$ – Thylacoleo Aug 20 '10 at 12:54
  • $\begingroup$ see below for answer... $\endgroup$ – Thylacoleo Aug 20 '10 at 13:11
11
$\begingroup$

I think I figured out the answer (to my own question). If the assumption of proportional hazards is true, the two methods give similar estimates of the hazard ratio. The discrepancy I found in one particular example, I now think, is due to the fact that that assumption is dubious.

If the assumption of proportional hazards is true, then a graph of log(time) vs. log(-log(St)) (where St is the proportional survival at time t) should show two parallel lines. Below is the graph created from the problem data set. It seems far from linear. If the assumption of proportional hazards is not valid, then the concept of a hazard ratio is meaningless, and so it doesn't matter which method is used to compute the hazard ratio.

alt text

I wonder if the discrepancy between the logrank and Mantel-Haenszel estimates of the hazard ratio can be used as a method to test the assumption of proportional hazards?

$\endgroup$
7
$\begingroup$

If I'm not mistaken, the log-rank estimator you reference is also known as the Pike estimator. I believe it's generally recommended for HR < 3 because it exhibits less bias in that range. The following paper may be of interest (note that the paper refers to it as O/E):

[...] The O/E method is biased but, within the range of values of the ratio of the hazard rates of interest in clinical trials, it is more efficient in terms of mean square error than either CML or the Mantel-Haenszel method for all but the largest trials. The Mantel-Haenszel method is minimally biased, gives answers very close to those obtained using CML, and may be used to provide satisfactory approximate confidence intervals.

$\endgroup$
  • $\begingroup$ Having had a brief look at that paper i'm not sure the estimates they consider are the same as those in the questioner's equations. I agree with the comments under the question - maybe in 1981 approximate methods were useful but these days there's no obvious reason not to use Cox regression. $\endgroup$ – onestop Aug 20 '10 at 12:42
  • $\begingroup$ @onestop: hmm, think definition of O/E == LR with the log forgotten above? I agree with what you say about Cox PH -- that's not the question I was trying to answer, but your advice is better in the broader context. $\endgroup$ – ars Aug 20 '10 at 15:27
  • $\begingroup$ Bernstein et. al. show some reasons (small n, ties) that cause the two methods to be inaccurate or different. But all of the discrepancies they showed are small. So I don't think anything in that paper explains the three fold discrepancy I saw that prompted this question. See below for the answer I came up with. $\endgroup$ – Harvey Motulsky Aug 23 '10 at 17:38
7
$\begingroup$

There are actually several more methods and the choice often depends on whether you are most interested in looking for early differences, later differences or - as for the log-rank test & the Mantel-Haenszel test - give equal weight to all time points.

To the question at hand. The log-rank test is in fact a form of the Mantel-Haenszel test applied to survival data. The Mantel-Haenszel test is usually used to test for independence in stratified contingency tables.

If we try to apply the MH test to survival data, we can start by assuming that events at each failure time are independent. We then stratify by failure time. We use the MH methods for by making each failure time a strata. Not surprisingly they often give the same result.

The exception occcurs when more than one event occurs simultaneously - multiple deaths at exactly the same time point. I can't remember how the treatment then differs. I think the log-rank test averages over the possible orderings of the tied failure times.

So the log-rank test is the MH test for survival data and can deal with ties. I've never used the MH test for survival data.

$\endgroup$
3
$\begingroup$

I thought I'd stumbled across a web site and reference that deals exactly with this question:

http://www.graphpad.com/faq/viewfaq.cfm?faq=1226 Start from "The two methods compared".

The site references the Berstein paper ars linked (above):

http://www.jstor.org/stable/2530564?seq=1

The site summarises Berstein et al's results nicely, so I'll quote it:

The two usually give identical (or nearly identical) results. But the results can differ when several subjects die at the same time or when the hazard ratio is far from 1.0.

Bernsetin and colleagues analyzed simulated data with both methods (1). In all their simulations, the assumption of proportional hazards was true. The two methods gave very similar values. The logrank method (which they refer to as the O/E method) reports values that are closer to 1.0 than the true Hazard Ratio, especially when the hazard ratio is large or the sample size is large.

When there are ties, both methods are less accurate. The logrank methods tend to report hazard ratios that are even closer to 1.0 (so the reported hazard ratio is too small when the hazard ratio is greater than 1.0, and too large when the hazard ratio is less than 1.0). The Mantel-Haenszel method, in contrast, reports hazard ratios that are further from 1.0 (so the reported hazard ratio is too large when the hazard ratio is greater than 1.0, and too small when the hazard ratio is less than 1.0).

They did not test the two methods with data simulated where the assumption of proportional hazards is not true. I have seen one data set where the two estimate of HR were very different (by a factor of three), and the assumption of proportional hazards was dubious for those data. It seems that the Mantel-Haenszel method gives more weight to differences in the hazard at late time points, while the logrank method gives equal weight everywhere (but I have not explored this in detail). If you see very different HR values with the two methods, think about whether the assumption of proportional hazards is reasonable. If that assumption is not reasonable, then of course the entire concept of a single hazard ratio describing the entire curve is not meaningful

The site also refer to the dataset in which "the two estimate of HR were very different (by a factor of three)", and suggest that the PH assumption is a key consideration.

Then I thought, "Who authored the site?" After a little searching I found it was Harvey Motulsky. So Harvey I've managed to reference you in answering your own question. You've become the authority!

Is the "problem dataset" a publicly available dataset?

$\endgroup$
  • $\begingroup$ I figured out the answer two days ago, and posted it here as a new answer. I also then expanded and updated the web page at graphpad.com that you found. I just edited that page again to include a link to an Excel file with the problem data (graphpad.com/faq/file/1226.xls). I couldn't do that until I got permission from the guy who generated the data (he wants to be anonymous, and the data is vaguely labeled). $\endgroup$ – Harvey Motulsky Aug 24 '10 at 18:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.