2
$\begingroup$

I keep running into situations (in my job) where I need to predict relatively rare events that occur at most once per entity, across many entities, and over time (e.g. predicting mortality of cancer patients based on treatments given).

I’ve gone through this process a few times applying both survival/hazard models as as well as cost-aware classifiers (or the myriad of other ways to bend a classifier to deal with imbalance) and found a few commonalities that I thought seemed to suggest an opportunity for research.

Basically, I find that classifiers work well in those scenarios and generally have more desirable properties but there is always one critical modeling question that has to be answered in order to cast survival problems as classification ones — over what time period in the future are you trying to predict the event in question?

To phrase this as an example, say I’m trying to predict when my car will break down based on how many miles it has and how many of the sensor/warning lights are on. As a survival regression problem, the objective is to estimate a probability curve for this failure time at some point in the future, but as a classification problem it must be reduced to a question like:

  1. “will my car break down in the next 1-3 weeks?”.
  2. “will it break down between 3 and 6 months from now?”
  3. “will it break down tomorrow?”

My point is that choosing this interval for classification isn’t easy and in the past I’ve just done it based on the classifier performance for a ton of different interval possibilities and tried to find some setting that gives the best performance.

My question then: Is anyone aware of a joint modeling strategy for both a binary classifier and the, let’s call it, “time-to-event” interval length? It seems like it would be possible with an MCMC sampler and an appropriate probability model, and I wanted to see if that kind of regression has already been tried before.

$\endgroup$
16
  • 2
    $\begingroup$ You can "read off" logistic models for given time intervals from a survival model, fit using e.g. Cox proportional hazards. $\endgroup$ Aug 18, 2015 at 14:30
  • $\begingroup$ On re-reading your question I'm a bit puzzled, because it's the application that mandates what you need to predict. Whether it's more useful to predict the probability that your car will break down tomorrow or the probability that your car will break down in the next three months depends on whether you're using your model to choose between driving & taking the bus tomorrow or to decide it's time to start looking for a new car. There then follows the interesting question of when "cutting" a survival model for a given time interval might give better predictions than fitting a logistic model ... $\endgroup$ Aug 18, 2015 at 14:56
  • $\begingroup$ ... to failure over a that time interval. So why are you choosing the interval? $\endgroup$ Aug 18, 2015 at 14:56
  • $\begingroup$ Ah thanks for giving it a read. I'm choosing it because the objective is simply to define probability of the event ever happening, which I find is only possible with reasonable accuracy for an interval that isn't too large (at some point, it's just too far in the future to predict well). This sounds like a survival regression problem then but the problem I have with that is that I have no way of verifying out of sample accuracy, or at least not one that seems widely accepted for proportional hazards models. I like those out of sample measures because they're easier to communicate, ... $\endgroup$
    – Eric Czech
    Aug 18, 2015 at 17:13
  • $\begingroup$ better for model selection, and I just plain trust them more than something like some in-sample information criteria score. So I lean towards classification where that's all better established but I really have no restrictions on the application side other than just "trying to predict things as far in advance as possible". That make sense? $\endgroup$
    – Eric Czech
    Aug 18, 2015 at 17:14

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.