# How do I determine if a survival model with missing data is appropriate?

Oversimplifying a bit, I have about a million records that record the entry time and exit time of people in a system spanning about ten years. Every record has an entry time, but not every record has an exit time. The mean time in the system is ~1 year.

The missing exit times happen for two reasons:

1. The person has not left the system at the time the data was captured.
2. The person's exit time was not recorded. This happens to say 50% of the records

The questions of interest are:

1. Are people spending less time in the system, and how much less time.
2. Are more exit times being recorded, and how many.

We can model this by saying that the probability that an exit gets recorded varies linearly with time, and that the time in the system has a Weibull whose parameters vary linearly with time. We can then make a maximum likelihood estimate of the various parameters and eyeball the results and deem them plausible. We chose the Weibull distribution because it seems to be used in measuring lifetimes and is fun to say as opposed to fitting the data better than say a gamma distribution.

Where should I look to get a clue as to how to do this correctly? We are somewhat mathematically savvy, but not extremely statistically savvy.

The basic way to see if your data is Weibull is to plot the log of cumulative hazards versus log of times and see if a straight line might be a good fit. The cumulative hazard can be found using the non-parametric Nelson-Aalen estimator. There are similar graphical diagnostics for Weibull regression if you fit your data with covariates and some references follow.

The Klein & Moeschberger text is pretty good and covers a lot of ground with model building/diagnostics for parametric and semi-parametric models (though mostly the latter). If you're working in R, Theneau's book is pretty good (I believe he wrote the survival package). It covers a lot of Cox PH and associated models, but I don't recall if it has much coverage of parametric models, like the one you're building.

BTW, is this a million subjects each with one entry/exit or recurrent entry/exit events for some smaller pool of people? Are you conditioning your likelihood to account for the censoring mechanism?

• Thanks, this is just what I was looking for. This is essentially a million subjects each with an entry and exit time. Yes we are conditioning to account for the censoring. Jul 27, 2010 at 12:03

You could use the estimated model to predict the exit times for all the people in your system. You could then compare the estimated exit times with the actual exit times (where you have this data) and compute a metric such as RMSE to assess how good your predictions are which will in turn give you a sense of model fit. See also this link.

• With a millon points and an 8 parameter model, a goodness of fit test like chi-squared tells me that there is essentially no chance that the model is correct. (Which is not surprising, as there are endless factors influencing reality that are not in the model) RMSE gives me a sense as to how good the model fits the data, but does not give me a sense of whether there is a better model Jul 27, 2010 at 1:07
• Well in order to find out if there is a better model, you could either experiment with different formulations or you could use various plots (e.g., exit times vs time) to see if the data is consistent with your model assumptions. You could also plot predicted exit times for a small sample selected at random vis-a-vis actual times to for model improvement ideas.
– user28
Jul 27, 2010 at 1:17