In log-rank testing how could the CI cross 1 with a p-value < .05?

Question

Came across these results comparing standard therapy with standard therapy and atezolizumab in endometrial cancer.

Findings state survival in placebo group vs treatment as HR 0·82, 95% CI 0·63–1·07; log-rank p=0·048.

Hence my question, under what circumstances in a log-rank test could the confidence interval contain 1 and yet yield a p-value of 0.048?

Answer 1

The paper says:

For survival endpoints, hazard ratios (HRs) are reported with 95% CIs based on a univariable Cox regression model... p values were based on the stratified log-rank test.

So the $p$-value isn't for the same test that was used to construct the confidence interval and it's quite possible for them to disagree slightly. I'd expect the stratified logrank test to be a bit more powerful than the unstratified Wald test corresponding to the confidence interval.

Answer 2

More generally. CI's and p-values are both approximations, and may well not be approximated by the same algorithms.

They are also not measuring the same thing -

• p-value is the probability that a null hypothesis of no-effect is true
• Confidence Interval is a range of values wide enough that (in this case) 95% of all estimates derived from doing the same study repeatedly will lie within it

(The implausibility of the latter is why most practicing scientists interpret CI's as 'Credible intervals')

See

• Disagreement between p-values and confidence intervals

and

• https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2689604/

for helpful introductory discussions of both of these points.