0
$\begingroup$

I am working on survival analysis and fit a model:

srFit <- survreg(formula = Surv(time) ~ f1 + f2 + f3 + f4 + f5 + f6 + f7 + f8 + f9 + f10, data = train, dist = dist_pred[i_dist])

Then I use the model to predict lifetime of test users:

pred <- predict(srFit, newdata=test, type='quantile', p=c(90 / 100))

But I don't really understand how quantile works. I know the $k^{th}$ quantile for a survival curve $S(t)$ is the location at which a horizontal line at height $p= 1-k$ intersects the plot of $S(t)$ and I know $S(t)$ is a decreasing function showing fraction of surviving members.

So my understanding is when $p=c(0.5)$ median is returned, let's say 20. Does this mean 20 is the point that probability of surviving is 0.5 ? If I use $p=c(0.9)$ then it will return the point that probability of surviving (ad not failing) is 0.9, right?

$\endgroup$
2
$\begingroup$

Survival regression models are based around a survivor function $S(t)$ which estimates, as a function of time, the proportion of people expected to remain surviving in the sample at that time.

In the documentation for type, interestingly, it says, "a predicted quantile on the original scale of the data "quantile"). So actually, it is predicting failure times. You can verify with a sanity check:

library(survival)
t <- rexp(1000)
fit <- survreg(Surv(t) ~ 1)
predict(fit, type='quantile', p=c(0, .25, .5, 1))[1, ]
quantile(t, c(0, .25, .5, 1)) ## empirical estimate of same values
$\endgroup$
  • $\begingroup$ Result is 0% 25% 50% 100% 0.001849746 0.298036777 0.686568095 6.588300066 Does this mean 25% fails at time 0.2980? What if I want to predict expected survival time? $\endgroup$ – Erin Oct 19 '15 at 17:23
  • $\begingroup$ @Erin if you have no censoring in your data, you can just use the empirical average from the survival times. However, if there are censored observations, your survival times will be biased. You would need to average up the probabilistic distribution estimated from the parametric survival model. $\endgroup$ – AdamO Oct 19 '15 at 18:16
1
$\begingroup$

The quantile function is the inverse of the Cumulative Distribution Function (CDF). Quantiles $q$ are inherently linked to probabilities of exceedance and return periods $T$. The mapping is given by: $q=1-\frac{1}{T}$. Intuitive examples arise when considering natural phenomena. For instance, the magnitude of the 500-year flood is the max stream flow value that is exceeded only once every 500 years. It corresponds to the $1-\frac{1}{500}=0.998$ quantile. Similarly, the 1000-year flood corresponds to the 0.999 quantile. The survival function $S$ is simply the complementary of the CDF $F$. That is, $S(x)=1-F(x)$.

The median value in your sample is given by the $q=0.5$ quantile. Half of your observations are smaller, and half are greater. For this value, half of your subjects are still alive. $q=0.9$ corresponds to the value $x_{0}$ of $X$ such that the CDF evaluated at this point is equal to $F(x_{0})=P[X\leq x_{0}]=0.9$., that is, $S(x_{0})=1-F(x_{0})=1-P[X\leq x_{0}]=1-0.9=0.1$. At this point, only 10% of your subjects are still alive. Beyond that point your system has a probability of failure of 0.9 (which is different from what you had in mind).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.