1
$\begingroup$

Concordance seems to be a very good metric for telling you how well the model allows you to order individuals for the particular sample that is being used. However, when applying a cox regression model with a single categorical variable (say male/female survival) to a population that is 99% female, the concordance will always come out at ~0.50 no matter how good the categorical variable (male/female) is at distinguishing people that are at high risk. Is there another way to measure how well a particular variable allows you to order two different groups, independent of the sample size that is used?

To illustrate the point, below is a small simulation to plot the concordance after fitting a cox proportional hazards model to some simulated data, where alpha is used to specify the proportion of individuals that are male/female. As noted, when alpha is small/large, the concordance is near 0.5.

conc <- c()
alphas <- seq(0,1,length.out=100)
for(alpha in alphas){
  n <- 1000
  x <- c(rep(0,round(n*alpha)),rep(1,round(n*(1-alpha))))
  deaths <- rexp(1000,rate=exp(5*x))
  # Assume no censoring
  sdata <- data.frame(Y=deaths,d=1,x=x)
  coxfit1 <- coxph(Surv(Y,d)~x,sdata)
  conc <- c(conc,as.numeric(summary(coxfit1)$concordance[1]))
}
plot(alphas,conc)

enter image description here

Improve this question
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

The concordance measure often used ($c$-index) is related to Somers' $D_{xy}$ rank correlation, which discounts ties on y but penalizes for ties on x. With a bad x distribution it is usually correct to penalize x this way. If you don't want to do that see if you can get a censored data variation on the Goodman-Kruskal $\gamma$ index. See the R Hmisc packge rcorr.cens function for some help.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks, this is helpful. I agree that it probably has a lot to do with how ties in hazard are included. However, I think overall, the concordance depends both on the size of the effects, and the proportions of the population. For instance, if you were considering a case where there were 3 groups, and 1 of the groups always had a much shorter lifetime than the other two, but consisted of only a small part of the population, and the other two groups were not distinct from one another, I don't think you would ever really be able to lift the concordance above 0.5. $\endgroup$
    – Clark
    Sep 3, 2015 at 19:30
  • $\begingroup$ Again, it depends on how you define relevant pairs. If you want to restrict X to not be tied within the pair, just like we do with Y, then select the option in rcorr.cens that mimics the Goodman-Kruskal $\gamma$. $\endgroup$
    – Frank Harrell
    Sep 3, 2015 at 19:36
0
$\begingroup$

The concordance measure being used was calculating concordance using: concordance = (agree + tied / 2.0) / (agree + disagree + tied) where tied counts those pairs of individuals with ties in their hazard. When the proportions are very unequal, most of the counts are in the tied category and hence the concordance is approx. 0.5.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Yes that's pretty much what we mentioned earlier. It's not ties in the hazard its ties in the linear predictor or any function of it. You could call this relative hazard or relative log hazard. And R Hmisc rcorr.cens has a simple workaround if you want to not penalize for ties in $X\beta$. This would not in general be reasonable if you want to claim that a variable with a lot of ties is a superior predictor to a continuous X. $\endgroup$
    – Frank Harrell
    Sep 4, 2015 at 16:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.