Residual Useful Life estimation from multivariate time series with lots of missing data and censoring, using neural networks

I have a set of industrial machines, for which I collect measurements from a few sensors (say, $d=20$) in time. So I have a $d-$variate time series for each machine. Let $t=0$ denote the start of my study: machine $j$ starts working at $t^j_{start}\geq 0$, and it either fails at $t^j_{fail}\geq t^j_{start}$, or it keeps working until the end of my study (so I have right-censored data). The machines work in very harsh environments, and often sensors measurement are temporarily not available. In other words, I have many missing data (say, about 50% of all sensor measurements are missing). Sensor time series are extremely long (say, one measurement/minute), but I can downsample them to daily measurements if necessary, maybe increasing $d$ to compensate (i.e., instead than $d$ instantaneous measurements per minute, $4d$ measurements for day, i.e., min, max, median and IQR for each sensor). After this downsampling, the lengths of the time series go from $50$ to $730^+$, more or less (i.e., the "worst" machines die after less than two months of operation, and the "best" ones are still running at the end of the study, after 2 years).

I would like to predict the Residual Useful Life, i.e., the remaining days until failure for a new machine. If $T$ is the useful life of a machine (thus a random variable) then

$$\text{RUL}(t)=\mathbf{E}[T|T\geq t]$$

In other words, let $\mathbf{x}(t)=(x_1(t),\dots,x_d(t))$ be the set of sensor measurements at time $t$ for the new machine, where possibly $x_i(t)=NA$ for one or more $i$: given in input the sequence $T=[\mathbf{x}(t_0),\dots,\mathbf{x}(t)]$, I want to estimate $\text{RUL}(t)\geq 0$, and as new data $\mathbf{x}(t)$ come in, I want to update my estimate $\text{RUL}(t)$.

I would like to use a Neural Network to do this: I was thinking of a Recurrent Neural Networks, because it's the architecture I'm more familiar with, for what it concerns time series forecasting, but I'm open to other suggestions, as long as open source implementations are available. I found this paper, but the authors are not willing to share code, and I feel the description is not detailed/clear enough to reimplement the code on my own. Can you help me? I'll also accept other suggestions.

PS for now I don't have static covariates for each machine, but in the future I'd like to include them in the analysis, things like for example the coordinates of the installation site of the machine, the type of the design of the machine, etc. In a "classical" survival analysis, these would correspond to things like height, weight, age, sex, etc. of each individual, while the sensor data would be for example time series of blood pressure, heart rate, body temperature, etc. Thus I'd like an approach which in the future can be extended by including such covariates.

• This problem is unclear to me. How do you define 'failure of a machine' and how does it relate to the data $\mathbf{x}(t)$? Do you want to predict 'when a sensor is gonna fail', 'when a sensor on a machine is gonna fail' or 'when a machine is gonna stop working'? Another point is that your data structure and gathering is not clear. For instance one of many questions is whether you have a one sensor for each of $d$ machines or $d$ sensors for each machine (of unspecified number). – Sextus Empiricus May 20 '18 at 12:19
• @martijinweterings you could have found the answers to some of your many questions, had you read my question more carefully. For example it's obvious that I have d sensors for each machine because I wrote it explicitly right at the beginning of my question: "So I have a d−variate time series for each machine". – DeltaIV May 20 '18 at 18:16

Two suggestions:

1. I recommend this reference for a nice example on building an ANN-based survival model. I do not think it handles NAs, so imputation might be needed.

2. If you think that the failure of a machine is not dependent on some instantaneous readings of some sensors, but rather on for example total amount of wear during a day you can use a flexible semiparametric model with time-dependent covariates. The same model can be used if you actually do know that a certain reading on a sensor (or a combination of readings) is associated with failure by including this covariate in the model. This model also allows using splines for including numerical covariates. Neither this model handles NAs.

I really recommend the second one as it is very easy to use and usually gives nice results. I have some experience with the second model I mention and can help you implementing it in R.

• Then I definitely recommend the second one, which I use daily and works like a charm. :) – Gino_JrDataScientist May 21 '18 at 10:20
• The documentation is somewhat patchy but I happen to work closely with the person who wrote the package, so I know a fair deal – Gino_JrDataScientist May 21 '18 at 10:24
• As for sudden failures, I would do two things: 1) inspect some failure events and see if there are any extreme readings/combination of readings that are statistically associated with the outcome. Unsure on the actual techniques I would use.. 2) get more info on the machines, from technicians or manuals. Then you can add the information gathered this way as additional covariates in a model. E.g. the manual states that the machine gets damaged at temperatures higher than 50C. Then include a variable that states whether the machine has passed 50C that day – Gino_JrDataScientist May 21 '18 at 10:33
• Regarding Egil's model - back several months ago I discussed with him and he said that most often the best predictor for failure is "how long has this machine run so far", basically in agreement with what we said, that accumulated wear is more important than instantaneous readings. So I would really try to include some covariates indicating the amount of stress exherted on the machine, probably as a function of some readings. This requires some domain knowledge, so again: ask technicians – Gino_JrDataScientist May 21 '18 at 10:38