Is it ever a good idea to give "partial credit" (continuous outcome) in training a logistic regression?

Question

I am training a logistic regression to predict which runners are most likely to finish a grueling endurance race.

Very few runners complete this race, so I have severe class imbalance and a small sample of successes (maybe a few dozen). I feel like I could get some good "signal" from the dozens of runners who almost made it. (My training data has not only completion, but also how far the ones that didn't finish actually made it.) So I am wondering whether it is a terrible idea or not to include some "partial credit." I came up with a couple functions for partial credit, the ramp and the logistic curve, which could be given various parameters.

The only difference with the regression would be that I would use training data to predict the modified, continuous outcome instead of a binary outcome. Comparing their predictions on a test set (using the binary response) I had fairly inconclusive results -- the logistic partial credit seemed to marginally improve R-squared, AUC, P/R, but this was just one attempt on one use case using a small sample.

I do not care about the predictions being uniformly biased toward completion -- what I care about is correctly ranking contestants on their likelihood to finish, or maybe even estimating their relative likelihood of finishing.

I understand that logistic regression assumes a linear relationship between predictors and the log of the odds ratio, and obviously this ratio has no real interpretation if I start messing with the outcomes. I'm sure this is not smart from a theoretical viewpoint, but it might help get some additional signal and prevent overfitting. (I have almost as many predictors as successes, so it may be helpful to use relationships with partial completion as a check on relationships with full completion).

Is this approach ever used in responsible practice?

Either way, are there other types of models out there (maybe something that explicitly models the hazard rate, applied over distance instead of time) that may be better suited for this type of analysis?

Answer 1

This seems like a job for survival analysis, like Cox proportional hazards analysis or possibly some parametric survival model.

Think about this problem in reverse from the way you're explaining it: what are the predictor variables associated with earlier distances to quitting?

Quitting is the event. The distance covered might be considered equivalent to time-to-event in standard survival analysis. You then have a number of events equal to the number of individuals who quit, so your problem with limited numbers of predictors will diminish. All those who quit provide information.

A Cox model, if it works on your data, will provide a linear predictor based on all the predictor variable values, ranking contestants in order of predicted distances to quitting.