The Stack Overflow podcast is back! Listen to an interview with our new CEO.
4 added 357 characters in body
source | link

I've spent a lot of time working with the general case of interval censoring, i.e., when the event time may be known exactly, right or left censored or only known up to an interval. For example, suppose a part is inspected and passed at $T_1$ and then inspected again at $T_2$ and failed. Then all we know is that it failed in the interval $(T_1, T_2]$.

In the interval censored case, while we can use bootstrap + asymptotic normality to make inference about the regression coefficients, this is not the case for the baseline survival curve itself. Thus, if one wants to make inference about actual survival times and not just hazard ratios, one needs to use the fully parametric model. As such, the semi-parametric model is often used more to check model fit rather than for full inference in regards to survival times.

Of course, this is not the case for right censored data. I would guess that the confidence intervals for the survival estimates are a bit tighter for a fully parametric model, although I have not tested that. In fact, see @AdamO's answer for more on that.  

FinallyAs another point, the AFT model does not have a semi-parametric model (in the sense of a Kaplan-Meier-like baseline distribution), even for right censored or uncensored data. Or more specifically, the model is very difficult to optimize. The reason for this is that you can think of the AFT model as rescaling the times, compared to the proportional hazards or odds models, which rescale the survival probabilities. The issue with this is that in a semi-parametric model, the only way in which event or censoring times affects the likelihood is the relative rank. Small enough movements of the event times will not change the ranks at all (assuming no ties in the data), meaning the derivatives are all zero without ties. And when there are ties, the derivatives are unbounded! Not a very fun optimization problem.

  Given that the AFT model is more resilient to missing covariates and more interpretable, there's a strong argument to use AFT, even though there is no semi-parametric model.

One more reason to favor parametric models over semi-parametric is that they can be easier to generalize. For example, if one wants to perform a Bayesian analysis, it's much easier with a parametric model. Or if one wants to build a cure-rate model, this is non-identifiable for a semi-parametric model, but is identifiable for a parametric model.

I've spent a lot of time working with the general case of interval censoring, i.e., when the event time may be known exactly, right or left censored or only known up to an interval. For example, suppose a part is inspected and passed at $T_1$ and then inspected again at $T_2$ and failed. Then all we know is that it failed in the interval $(T_1, T_2]$.

In the interval censored case, while we can use bootstrap + asymptotic normality to make inference about the regression coefficients, this is not the case for the baseline survival curve itself. Thus, if one wants to make inference about actual survival times and not just hazard ratios, one needs to use the fully parametric model. As such, the semi-parametric model is often used more to check model fit rather than for full inference in regards to survival times.

Of course, this is not the case for right censored data. I would guess that the confidence intervals for the survival estimates are a bit tighter for a fully parametric model, although I have not tested that. In fact, see @AdamO's answer for more on that.  

Finally, the AFT model does not have a semi-parametric model (in the sense of a Kaplan-Meier-like baseline distribution), even for right censored or uncensored data. Or more specifically, the model is very difficult to optimize. The reason for this is that you can think of the AFT model as rescaling the times, compared to the proportional hazards or odds models, which rescale the survival probabilities. The issue with this is that in a semi-parametric model, the only way in which event or censoring times affects the likelihood is the relative rank. Small enough movements of the event times will not change the ranks at all (assuming no ties in the data), meaning the derivatives are all zero without ties. And when there are ties, the derivatives are unbounded! Not a very fun optimization problem.

  Given that the AFT model is more resilient to missing covariates and more interpretable, there's a strong argument to use AFT, even though there is no semi-parametric model.

I've spent a lot of time working with the general case of interval censoring, i.e., when the event time may be known exactly, right or left censored or only known up to an interval. For example, suppose a part is inspected and passed at $T_1$ and then inspected again at $T_2$ and failed. Then all we know is that it failed in the interval $(T_1, T_2]$.

In the interval censored case, while we can use bootstrap + asymptotic normality to make inference about the regression coefficients, this is not the case for the baseline survival curve itself. Thus, if one wants to make inference about actual survival times and not just hazard ratios, one needs to use the fully parametric model. As such, the semi-parametric model is often used more to check model fit rather than for full inference in regards to survival times.

Of course, this is not the case for right censored data. I would guess that the confidence intervals for the survival estimates are a bit tighter for a fully parametric model, although I have not tested that. In fact, see @AdamO's answer for more on that.

As another point, the AFT model does not have a semi-parametric model (in the sense of a Kaplan-Meier-like baseline distribution), even for right censored or uncensored data. Or more specifically, the model is very difficult to optimize. The reason for this is that you can think of the AFT model as rescaling the times, compared to the proportional hazards or odds models, which rescale the survival probabilities. The issue with this is that in a semi-parametric model, the only way in which event or censoring times affects the likelihood is the relative rank. Small enough movements of the event times will not change the ranks at all (assuming no ties in the data), meaning the derivatives are all zero without ties. And when there are ties, the derivatives are unbounded! Not a very fun optimization problem. Given that the AFT model is more resilient to missing covariates and more interpretable, there's a strong argument to use AFT, even though there is no semi-parametric model.

One more reason to favor parametric models over semi-parametric is that they can be easier to generalize. For example, if one wants to perform a Bayesian analysis, it's much easier with a parametric model. Or if one wants to build a cure-rate model, this is non-identifiable for a semi-parametric model, but is identifiable for a parametric model.

3 added 60 characters in body
source | link

I've spent a lot of time working with the general case of interval censoring, i.e., when the event time may be known exactly, right or left censored or only known up to an interval. For example, suppose a part is inspected and passed at $T_1$ and then inspected again at $T_2$ and failed. Then all we know is that it failed in the interval $(T_1, T_2]$.

In the interval censored case, while we can use bootstrap + asymptotic normality to make inference about the regression coefficients, this is not the case for the baseline survival curve itself. Thus, if one wants to make inference about actual survival times and not just hazard ratios, one needs to use the fully parametric model. As such, the semi-parametric model is often used more to check model fit rather than for full inference in regards to survival times.

Of course, this is not the case for right censored data. I would guess that the confidence intervals for the survival estimates are a bit tighter for a fully parametric model, although I have not tested that. In fact, see @AdamO's answer for more on that.

Finally, the AFT model does not have a semi-parametric model (in the sense of a Kaplan-Meier-like baseline distribution), even for right censored or uncensored data. Or more specifically, the model is very difficult to optimize. The reason for this is that you can think of the AFT model as rescaling the times, compared to the proportional hazards or odds models, which rescale the survival probabilities. The issue with this is that in a semi-parametric model, the only way in which event or censoring times affects the likelihood is the relative rank. Small enough movements of the event times will not change the ranks at all (assuming no ties in the data), meaning the derivatives are all zero without ties. And when there are ties, the derivatives are unbounded! Not a very fun optimization problem.

Given that the AFT model is more resilient to missing covariates and more interpretable, there's a strong argument to use AFT, even though there is no semi-parametric model.

I've spent a lot of time working with the general case of interval censoring, i.e., when the event time may be known exactly, right or left censored or only known up to an interval. For example, suppose a part is inspected and passed at $T_1$ and then inspected again at $T_2$ and failed. Then all we know is that it failed in the interval $(T_1, T_2]$.

In the interval censored case, while we can use bootstrap + asymptotic normality to make inference about the regression coefficients, this is not the case for the baseline survival curve itself. Thus, if one wants to make inference about actual survival times and not just hazard ratios, one needs to use the fully parametric model. As such, the semi-parametric model is often used more to check model fit rather than for full inference in regards to survival times.

Of course, this is not the case for right censored data. I would guess that the confidence intervals for the survival estimates are a bit tighter for a fully parametric model, although I have not tested that. In fact, see @AdamO's answer for more on that.

Finally, the AFT model does not have a semi-parametric model, even for right censored or uncensored data. Or more specifically, the model is very difficult to optimize. The reason for this is that you can think of the AFT model as rescaling the times, compared to the proportional hazards or odds models, which rescale the survival probabilities. The issue with this is that in a semi-parametric model, the only way in which event or censoring times affects the likelihood is the relative rank. Small enough movements of the event times will not change the ranks at all (assuming no ties in the data), meaning the derivatives are all zero without ties. And when there are ties, the derivatives are unbounded! Not a very fun optimization problem.

Given that the AFT model is more resilient to missing covariates and more interpretable, there's a strong argument to use AFT, even though there is no semi-parametric model.

I've spent a lot of time working with the general case of interval censoring, i.e., when the event time may be known exactly, right or left censored or only known up to an interval. For example, suppose a part is inspected and passed at $T_1$ and then inspected again at $T_2$ and failed. Then all we know is that it failed in the interval $(T_1, T_2]$.

In the interval censored case, while we can use bootstrap + asymptotic normality to make inference about the regression coefficients, this is not the case for the baseline survival curve itself. Thus, if one wants to make inference about actual survival times and not just hazard ratios, one needs to use the fully parametric model. As such, the semi-parametric model is often used more to check model fit rather than for full inference in regards to survival times.

Of course, this is not the case for right censored data. I would guess that the confidence intervals for the survival estimates are a bit tighter for a fully parametric model, although I have not tested that. In fact, see @AdamO's answer for more on that.

Finally, the AFT model does not have a semi-parametric model (in the sense of a Kaplan-Meier-like baseline distribution), even for right censored or uncensored data. Or more specifically, the model is very difficult to optimize. The reason for this is that you can think of the AFT model as rescaling the times, compared to the proportional hazards or odds models, which rescale the survival probabilities. The issue with this is that in a semi-parametric model, the only way in which event or censoring times affects the likelihood is the relative rank. Small enough movements of the event times will not change the ranks at all (assuming no ties in the data), meaning the derivatives are all zero without ties. And when there are ties, the derivatives are unbounded! Not a very fun optimization problem.

Given that the AFT model is more resilient to missing covariates and more interpretable, there's a strong argument to use AFT, even though there is no semi-parametric model.

2 added 52 characters in body
source | link

I've spent a lot of time working with the general case of interval censoring, i.e., when the event time may be known exactly, right or left censored or only known up to an interval. For example, suppose a part is inspected and passed at $T_1$ and then inspected again at $T_2$ and failed. Then all we know is that it failed in the interval $(T_1, T_2]$.

In theIn the interval censored case, while we can use bootstrap + asymptotic normality to make inference about the regression coefficients, this is not the case for the baseline survival curve itself. Thus, if one wants to make inference about actual survival times and not just hazard ratios, one needs to use the fully parametric model. As such, the semi-parametric model is often used more to check model fit rather than for full inference in regards to survival times.

Of course, this is not the case for right censored data. I would guess that the confidence intervals for the survival estimates are a bit tighter for a fully parametric model, although I have not tested that. In fact, see @AdamO's answer for more on that.

Finally, the AFT model does not have a semi-parametric model, even for right censored or uncensored data. Or more specifically, the model is very difficult to optimize. The reason for this is that you can think of the AFT model as rescaling the times, compared to the proportional hazards or odds models, which rescale the survival probabilities. The issue with this is that in a semi-parametric model, the only thing thatway in which event or censoring times affects the likelihood is the order of the datarelative rank. Small enough movements of the event times will not change the ranks at all (assuming no ties in the data), meaning the derivatives are all zero without ties. And when there are ties, in which case the derivatives are unbounded.! Not a very fun optimization problem.

Given that the AFT model is more resilient to missing covariates and much more interpretable, there's a strong argument to use AFT, even though there is no semi-parametric model!.

I've spent a lot of time working with the general case of interval censoring, i.e., when the event time may be known exactly, right or left censored or only known up to an interval. For example, suppose a part is inspected and passed at $T_1$ and then inspected again at $T_2$ and failed. Then all we know is that it failed in the interval $(T_1, T_2]$.

In the interval censored case, while we can use bootstrap + asymptotic normality to make inference about the regression coefficients, this is not the case for the baseline survival curve itself. Thus, if one wants to make inference about actual survival times and not just hazard ratios, one needs to use the fully parametric model. As such, the semi-parametric model is often used more to check model fit rather than for full inference.

Of course, this is not the case for right censored data. I would guess that the confidence intervals for the survival estimates are a bit tighter for a fully parametric model, although I have not tested that.

Finally, the AFT model does not have a semi-parametric model, even for right censored or uncensored data. Or more specifically, the model is very difficult to optimize. The reason for this is that you can think of the AFT model as rescaling the times, compared to the proportional hazards or odds models, which rescale the survival probabilities. The issue with this is that in a semi-parametric model, the only thing that affects the likelihood is the order of the data. Small enough movements of the event times will not change the ranks at all (assuming no ties in the data), meaning the derivatives are all zero without ties, in which case the derivatives are unbounded. Not a very fun optimization problem.

Given that the AFT model is more resilient to missing covariates and much more interpretable, there's a strong argument to use AFT, even though there is no semi-parametric model!

I've spent a lot of time working with the general case of interval censoring, i.e., when the event time may be known exactly, right or left censored or only known up to an interval. For example, suppose a part is inspected and passed at $T_1$ and then inspected again at $T_2$ and failed. Then all we know is that it failed in the interval $(T_1, T_2]$.

In the interval censored case, while we can use bootstrap + asymptotic normality to make inference about the regression coefficients, this is not the case for the baseline survival curve itself. Thus, if one wants to make inference about actual survival times and not just hazard ratios, one needs to use the fully parametric model. As such, the semi-parametric model is often used more to check model fit rather than for full inference in regards to survival times.

Of course, this is not the case for right censored data. I would guess that the confidence intervals for the survival estimates are a bit tighter for a fully parametric model, although I have not tested that. In fact, see @AdamO's answer for more on that.

Finally, the AFT model does not have a semi-parametric model, even for right censored or uncensored data. Or more specifically, the model is very difficult to optimize. The reason for this is that you can think of the AFT model as rescaling the times, compared to the proportional hazards or odds models, which rescale the survival probabilities. The issue with this is that in a semi-parametric model, the only way in which event or censoring times affects the likelihood is the relative rank. Small enough movements of the event times will not change the ranks at all (assuming no ties in the data), meaning the derivatives are all zero without ties. And when there are ties, the derivatives are unbounded! Not a very fun optimization problem.

Given that the AFT model is more resilient to missing covariates and more interpretable, there's a strong argument to use AFT, even though there is no semi-parametric model.

1
source | link