1
$\begingroup$

As mentioned in this comment and answer How to get fitted values from clogit model, it is not clear that predicting from a conditional logistic regression model is meaningful.

It seems to me that it's even less meaningful to calculate AUC (or, worse, cross-validated AUC) from such a model. Conditional logistic regression is a relative risk model: conditional on participants being in the same stratum, this is how log-odds risk is related to the covariates. It doesn't seem to make sense to use the covariate coefficients as absolute risks, calculate expected risk, and calculate AUC. Particularly for cross-validated AUC -- since we don't split strata across folds, by definition, we're taking a model built on certain strata, and attempting to predict on other strata.

So, what tests of model performance do we have?

  1. I'd be interested in global goodness-of-fit tests, similar to various likelihood-related tests.

  2. But more interested in tests of model predictive ability, since the underlying research question (in my particular case) is what sorts of covariates lead to better predictive models.

$\endgroup$

1 Answer 1

0
$\begingroup$

I focus on the ROC as an metric for discrimination ability of prediction model.

In the general ROC estimation, we should build a logistic regression model and estimate unconditional probability of a positive response for each observation. For example:

set.seed(63126)
n <- 1000
x <- rnorm(n)
pr <- exp(x)/(1+exp(x))
y <- 1*(runif(n) < pr)
mod <- glm(y~x, family="binomial")
predpr <- predict(mod,type=c("response"))
library(pROC)
roccurve <- roc(y ~ predpr)
plot(roccurve)

However, in the conditional logistic regression, we only get the probability of a particular one will have a positive response among the group of observations that this observation belongs to (strata). So we can not plot the ROC directly.

In 2020, Hui Xu et al. introduced an alternative to solve this problem, the algorithm has not been integrated in SAS, R and Stata.

Xu H, Qian J, Paynter NP, et al. Estimating the receiver operating characteristic curve in matched case control studies. Stat Med. 2019;38(3):437-451. doi:10.1002/sim.7986

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.