I am training a logistic regression to predict which runners are most likely to finish a grueling endurance race.
Very few runners complete this race, so I have severe class imbalance and a small sample of successes (maybe a few dozen). I feel like I could get some good "signal" from the dozens of runners who almost made it. (My training data has not only completion, but also how far the ones that didn't finish actually made it.) So I am wondering whether it is a terrible idea or not to include some "partial credit." I came up with a couple functions for partial credit, the ramp and the logistic curve, which could be given various parameters.
The only difference with the regression would be that I would use training data to predict the modified, continuous outcome instead of a binary outcome. Comparing their predictions on a test set (using the binary response) I had fairly inconclusive results -- the logistic partial credit seemed to marginally improve R-squared, AUC, P/R, but this was just one attempt on one use case using a small sample.
I do not care about the predictions being uniformly biased toward completion -- what I care about is correctly ranking contestants on their likelihood to finish, or maybe even estimating their relative likelihood of finishing.
I understand that logistic regression assumes a linear relationship between predictors and the log of the odds ratio, and obviously this ratio has no real interpretation if I start messing with the outcomes. I'm sure this is not smart from a theoretical viewpoint, but it might help get some additional signal and prevent overfitting. (I have almost as many predictors as successes, so it may be helpful to use relationships with partial completion as a check on relationships with full completion).
Is this approach ever used in responsible practice?
Either way, are there other types of models out there (maybe something that explicitly models the hazard rate, applied over distance instead of time) that may be better suited for this type of analysis?