Suppose that we monitor a population of devices over an interval $[a,b]$. Some devices are added before $a$ and some are added during $[a,b]$. Furthermore, some devices fail during $[a,b]$ and some are healthy until time $b$ and their ages are right censored. Roughly, $90\%$ of the population are right censored. We can assume that the devices are added randomly. We know all device ages during $[a,b]$.

Is this an example of a random censoring? I'm using the following type definitions from Wikipedia: https://en.wikipedia.org/wiki/Censoring_(statistics)#Types

We are interested to perform survival analysis to estimate the failure distribution $f(t)$ of these devices. My intuition is that since many of the samples are right censored, our estimation of the tail of distribution $f(t)$ will be inaccurate and biased.

Now, suppose that we are only interested in estimating $f(t)$ over the interval $[0,c]$ where $c < b - a$. Is there a way to obtain an unbiased estimate of $f(t)$ over this interval? Also, we have a lot of data so it is okay to only use a fraction of them.

  • $\begingroup$ Could/did any devices fail before $a$? $\endgroup$ – Cam.Davidson.Pilon Apr 18 '19 at 2:57
  • $\begingroup$ We don't have such information in the data. @Cam.Davidson.Pilon $\endgroup$ – KRL Apr 18 '19 at 3:01
  • 1
    $\begingroup$ For the devices added prior to $a$, do you know when they were added? $\endgroup$ – Cam.Davidson.Pilon Apr 18 '19 at 3:08
  • $\begingroup$ @Cam.Davidson.Pilon Yes, we know when they were added. We basically know all device ages in the interval [a,b]. $\endgroup$ – KRL Apr 18 '19 at 3:15
  • 1
    $\begingroup$ Could you please add the fact that you know all the device ages to the original question? That's important to the answer; comments sometimes get deleted in which case that information would be lost. $\endgroup$ – EdM Apr 18 '19 at 15:05

To answer your first question, it sounds like random censoring since the censoring time, $b$, is constant for everyone, hence statistically independent of true failure times (How $b$ is chosen may be important, but likely not).

Though you have lots of right-censored data, you may still have information about the tail. If you use a parametric model, this can give you the entire distribution, even from data that can't be larger than $b$. If you use a non-parametric model, then you are limited to inferences up to your max observation time.

Here's an example. Suppose your data looks like the following, with $a=10, b=20$.

enter image description here

A red line is the lifetimes of an observed failure, and a blue line is a censoring event. The black lines denote our observation period, $[10, 20]$.

Below is the Kaplan-Meier curve for the above data:

enter image description here

Our inference is limited by the largest uncensored observation, which is 14 (=19-5).

We'll switch to a parametric model. Let's try an Weibull distribution. Below we can see that we can extrapolate to beyond our largest observation (but keep in mind, this is walking on thin ice):

enter image description here

Now, reading your second question, I'm a bit confused. I understood $a,b$ to be wall clock times (see my picture above). But you mention you want inference for $[0, c]$, which doesn't make sense to me. ($[a, b]$ can't be relative to subject time, because you mention adding devices in that time interval).

| cite | improve this answer | |
  • $\begingroup$ This explains how to do the Kaplan-Meier (KM) estimate (given that the actual lifetimes of all cases with observed failures were known, as they are in this case) but doesn't get to the question of bias in the estimate. That might be an issue with 90% censoring. There is an R package that implements a jackknifing method to reduce bias in KM estimates, if that's of interest. $\endgroup$ – EdM Apr 18 '19 at 15:13
  • $\begingroup$ Your interpretation of [a,b] is correct. In the second question, we only care about having a good estimate (less biased) of survival function over [0,c] instead of [0,inf]. I think it should be easier to get a better estimate of survival function over [0,c] because it is less affected by the censoring. For example, in KM estimate, the confidence band is narrower in the beginning. $\endgroup$ – KRL Apr 18 '19 at 19:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.