I'm looking for a principled way to estimate when an event of interest is overdue (a binary decision/alert), not just predicting when it is supposed to happen.
In the survival analysis literature I have not come across this particular question yet: the typical survival paper/algorithm is about estimating 'how much longer' until an event occurs -- which is never going to be overdue, because by construction it's always in the future. While this makes sense mathematically/probabilistically/conceptually, it's not helping to define formally what it means for an event to be overdue and how one could alert on this based on a probabilistic framework.
As a concrete example, consider the problem of alerting doctors when the patient's recovery process goes 'off track', i.e., alert them when recovery is overdue (vs potential complications looming). A doctor at "t=0" (day of surgery) has their own predictive distribution for recovery time based on personal experience, and tells patient that they are going to check in again at some point in the future (say, the 80% quantile). Yet, if patient has a heart rate & body temperature monitor on (time-varying features), then every moment a predictive model could tell us the conditional distribution of recovery given heart rate & body temperature. This conditional distribution is always in the future and it moves further into the future with every day passing by some $\Delta > 0$ (conditional survival curves).
Considerations so far:
take the prediction interval from a t=0, and alert when observed duration is outside the interval: that works assuming as long as I don't have access to time-varying features. In doctor example, if I have a data feed on "blood pressure", "cholesterol", and "body temperature", then ideally I want to take this information into account for alerting, not rely on information at t=0. But I don't just want to rely on doctor telling me "when cholesterol gets higher than 'X' we should check in again" -- rather rely on models to forecast that recovery pattern [and augment doctor's opinion].
take prediction interval from 'yesterday': that predictive distribution [using 'blood pressure' and 'cholesterol' features], will in practice have the predictive distribution far into the future -- no matter how long we already go without recovery -- with a $Pr(\text{time until recovery} \leq 24hrs) \approx 0$.
don't predict remaining time, but always predict "total time": that solves the problem as now 'overdue' is just when total prediction is less than observed total. However, not a fan of this as I have a model predicting already (partially) observed events (+ inconsistencies when using a total duration model make statements about future duration).
re-frame as 'determine best snapshot to use as baseline' problem: find the best snapshot s, 0 < s < t (today), for which it makes sense to consider features up to "s", and then use the prediction interval from $Pr_s(\text{time until recovery} \mid \text{features at day s}, \text{survived s days})$ and compare it to actual observed duration since "s", t - s. Alert when that probability is above a certain cutoff (say, 90%). That does not really solve the problem, but just moves it out of the traditional survival analysis framework.
One idea here is to use that snapshot 's', where the conditional survival curve based on forecast at time 's' given it has survived until today is close to the current predictive distribution using all features up to time 't'. I.e. start today at 't' and move it back as far as possible as long as the distance (KL divergence?) between $Pr_t(\text{remaining duration} \mid \text{features at time t})$ and $Pr_s(\text{ remaining duration} \mid \text{ features at day s, survived another t-s days})$ is "small". Determine what is a good "small" threshold by backtesting and using empirical null hypothesis estimation (Efron). Then use the prediction interval from this optimal s*, compare to observed duration (t - s*), and alert if outside interval.Treat as a causal inference problem, where "alert" is the treatment variable, duration until recovery is the outcome variable, and now it's a classic causal inference problem to predict unit level treatment effect (UTE) and pick the one where it has highest impact in reducing duration (to recovery) -- e.g., Kelly, Kong, and Goerg (2022) for a UTE causal ML algorithm. Issue is that we don't have historically observed this 'alert' as a treatment; alerting is a new concept that we haven't measured historically.
What I want to provide the doctor/hospital is a statistical/ML system to page the doctor when patient is 'overdue' that takes time-varying covariates into account but does not trivially never alert, because based on current model predictions recovery is always due (far) in the future. Looking for any references to existing work (maybe I'm just searching for the wrong keywords/terminology to find papers that talk about this problem) or pragmatic ideas that can be implemented using estimated survival predictive distributions (e.g., from Weibull RNNs).
Update 2022/10/31: I removed the truck maintenance example from original post. My use case is the doctor example, where it's of interest to make the event happen quickly -- and catch early when things go "off track".
