13
$\begingroup$

I know that traditional statistical models like Cox Proportional Hazards regression & some Kaplan-Meier models can be used to predict days till next occurrence of an event say failure etc. i.e Survival analysis

Questions

  1. How can regression version of machine learning models like GBM, Neural networks etc be used to predict days till occurrence of an event?
  2. I believe just using days till occurence as target variable and simplying running a regression model will not work? Why wont it work & how can it be fixed?
  3. Can we convert the survival analysis problem to a classification and then obtain survival probabilities? If then how to create the binary target variable?
  4. What is the pros & cons of machine learning approach vs Cox Proportional Hazards regression & Kaplan-Meier models etc?

Imagine sample input data is of the below format

enter image description here

Note:

  • The sensor pings the data at intervals of 10 mins but at times data can be missing due to network issue etc as represented by the row with NA.
  • var1,var2,var3 are the predictors, explanatory variables.
  • failure_flag tells whether the machine failed or not.
  • We have last 6 months data at every 10 min interval for each of the machine ids

EDIT:

Expected output prediction should be in the below format enter image description here

Note: I want to predict the probability of failure for each of the machines for the next 30 days at daily level.

$\endgroup$
  • 1
    $\begingroup$ I think it would help if you could explain why this is time-to-event data; what, exactly, is the response you want to model? $\endgroup$ – Cliff AB Mar 2 '16 at 17:44
  • $\begingroup$ I have edited and added the expected output prediction table to make it clear. Let me know if you have any further questions. $\endgroup$ – GeorgeOfTheRF Mar 2 '16 at 18:00
  • 1
    $\begingroup$ There are ways of converting survival data to binary outcomes in some cases, e.g., discrete time hazard models: statisticalhorizons.com/wp-content/uploads/Allison.SM82.pdf. Some machine learning methods such as random forests can model time to event data by, for instance, using the log rank statistic as the splitting criterion. $\endgroup$ – dsaxton Mar 2 '16 at 18:14
  • $\begingroup$ @dsaxton Thanks. Can you explain how to conver the above survival data to binary outcomes? $\endgroup$ – GeorgeOfTheRF Mar 2 '16 at 18:21
  • $\begingroup$ After taking a closer look it seems you already have binary outcomes with the failure_flag. $\endgroup$ – dsaxton Mar 2 '16 at 21:34
6
$\begingroup$

For the case of neural networks, this is a promising approach: WTTE-RNN - Less hacky churn prediction.

The essence of this method is to use a Recurrent Neural Network to predict parameters of a Weibull distribution at each time-step and optimize the network using a loss function that takes censoring into account.

The author also released his implementation on Github.

$\endgroup$
2
$\begingroup$

Have a look at these references:

https://www.stats.ox.ac.uk/pub/bdr/NNSM.pdf

http://pcwww.liv.ac.uk/~afgt/eleuteri_lyon07.pdf

Also note that traditional hazards-based models like Cox Proportional Hazards (CPH) are not designed to predict time-to-event, but rather to infer variables' impact (correlation) against i) observations of events and, hence ii) a survival curve. Why? Look at the CPH's MLE.

Hence, if you want to more directly predict something like "days till occurrence", CPH may not be advisable; other models may better serve your task as noted in the above two references.

$\endgroup$
1
$\begingroup$

As @dsaxton said, you can build a discrete time model. You set it up to predict p(fail at this day given survived up to previous day). Your inputs are current day (in whatever representation you want) eg one hot encoding, integer,.. Spline... As well as any other independent variables you might want

So you create rows of data, for each sample that survived till time t-1, did it die at time t (0/1).

So now the probability of surviving up to time T is the product of p(don't die at time t given didn't die at t-1) for t=1 to T. Ie you make T predictions from your model and then multiply together.

I would say the reason its not such an idea to directly predict time to failure is because of the hidden structure of the problem. Eg what do you input for machines that didn't fail. The underlying structure is effectively the independent events: fail at time t given didn't fail up to t-1. So eg if you assume it is constant, then your survival curve becomes an exponential (see hazard models)

Note in you case you could model at 10 minute interval or aggregate up the classification problem up to day level..

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.