# In survival analysis, when should we use fully parametric models over semi-parametric ones?

This question is the counterpoint of the other question In survival analysis, why do we use semi-parametric models (Cox proportional hazards) instead of fully parametric models?

Indeed, it clearly demonstrates the advantages of Cox Proportional Hazards regression over fully parametric ones, without assumption on the distribution of the survival time.

Still, there are some recent R packages (SmoothHazard(2017) for instance, function shr with method="Weib") which makes it possible to easily fit fully parametric models.

I happen to have had the opportunity to perform both on a 50k dataset, with very similar results.

What benefits are expected from a fully parametric survival model? What additional analyzes would it allow?

This has been studied in detail for many years and there is a large literature. I really like spline hazard models. The simplest answer to your question is this:

• If you want to estimate covariate effects, especially in the absence of time-dependent covariates, then semiparametric models such as the Cox proportional hazards model are usually preferred because they are fast, robust, and Y-transformation invariant
• Flexible parametric models are a little more efficient for estimating absolute quantities such as survival curves
• Parametric models provide a formula that makes prediction easier
• If you can integrate the hazard function analytically when time-dependent covariates are present, parametric models provide faster prediction and more intuition
• Parametric models can extrapolate (but beware) to yield survival estimates beyond the last follow-up time, and to estimate expected (mean) survival time

In summary I'd say the main reason to like parametric survival models is not efficiency, but rather ease of interpretation and of obtaining predictions for future observations.

See this paper for example.

• Great answer thanks. I've been wondering about the spline hazard model, the SmoothHazard package's shr vignette says that it can use a "linear combination of M-splines", which would thus be semi-parametric but makes no assumption on the hazard functional form. The paper you linked is about RCS, does it make such assumption ? – Dan Chaltiel Mar 27 '19 at 14:47

When you know the actual functional form of the hazard function, the fully parametric survival model is far more efficient than the Cox model. Statistical efficiency is like power. A good way to think of it is the width of the confidence interval for your final estimate of the log-hazard ratios: a tight CI is the result of an efficient analysis (assuming you have an unbiased estimator).

Exponential and Weibull survival models are indeed popular examples of "known" hazard functions (constant and linear in time respectively). But you could have any old baseline hazard function $$\lambda(t)$$, and calculate the expected survival at any time for any combination of covariates given a parameter estimate $$\theta$$ as:

$$S(\theta, t) = \exp(\Lambda(t)\exp(\theta \mathbf{X}))$$

where $$\Lambda(t)$$ is the cumulative hazard. An iterative EM-type solver would lead to maximum likelihood estimates of $$\theta$$.

It is a neat fact that, assuming a constant hazard, the relatively efficiency of the Cox model to the Weibull model to the Exponential fully parametric survival model is 3:2:1. That is, when the data are actually exponential, it will take 9 times as many observations under a Cox model to produce a confidence interval for the effect estimate, $$\theta$$ with an equal expected half-width as that of the exponential survival model. You must use what you know when you know it, but never assume wrongly.

• Great answer thanks. But how could someone know the hazard function's functional form? – Dan Chaltiel Mar 27 '19 at 14:48
• @DanChaltiel it's a paradox if ever there was one. Prior research or heuristic models could be used to support the idea. For instance, emission of radioactive isotopes have independent interarrival times. Some diagnostics like plots of studentized residuals vs. event time could be used to see if the (fully parametric) model is appropriate. Estimating the hazard with a kernal smoother speaks to fharrel's answer though I suspect the flexibility/generalizatbility tradeoff leads to results asymptotically consistent with Cox models at their most flexible. – AdamO Mar 27 '19 at 15:23
• In calculations I did a long time ago the efficiency of parametric models is not nearly as great as @AdamO found, when estimating survival probabilities. In terms of estimating hazard ratios, the Cox model is exactly as efficient as the simple exponential model. – Frank Harrell Mar 27 '19 at 19:50
• Example: if you look at confidence intervals for the empirical cumulative distribution function without censoring, they are about the same width as you get from estimating 3-4 parameters of a specific parametric distribution. – Frank Harrell Mar 27 '19 at 20:25
• @FrankHarrell: probably not surprisingly, I think the relative efficiency of the Cox PH vs a parametric model for survival probabilities is highly sensitive to both the distribution of the censoring and which quantile you are estimating. As a really extreme case, suppose you have some data that is right censored and the last inspection time is beyond the last event time. Then tail of the semi-parametric model is non-identifiable, while the parameteric model (for better or worse) is! – Cliff AB Mar 28 '19 at 18:01

I've spent a lot of time working with the general case of interval censoring, i.e., when the event time may be known exactly, right or left censored or only known up to an interval. For example, suppose a part is inspected and passed at $$T_1$$ and then inspected again at $$T_2$$ and failed. Then all we know is that it failed in the interval $$(T_1, T_2]$$.

In the interval censored case, while we can use bootstrap + asymptotic normality to make inference about the regression coefficients, this is not the case for the baseline survival curve itself. Thus, if one wants to make inference about actual survival times and not just hazard ratios, one needs to use the fully parametric model. As such, the semi-parametric model is often used more to check model fit rather than for full inference in regards to survival times.

Of course, this is not the case for right censored data. I would guess that the confidence intervals for the survival estimates are a bit tighter for a fully parametric model, although I have not tested that. In fact, see @AdamO's answer for more on that.

As another point, the AFT model does not have a semi-parametric model (in the sense of a Kaplan-Meier-like baseline distribution), even for right censored or uncensored data. Or more specifically, the model is very difficult to optimize. The reason for this is that you can think of the AFT model as rescaling the times, compared to the proportional hazards or odds models, which rescale the survival probabilities. The issue with this is that in a semi-parametric model, the only way in which event or censoring times affects the likelihood is the relative rank. Small enough movements of the event times will not change the ranks at all (assuming no ties in the data), meaning the derivatives are all zero without ties. And when there are ties, the derivatives are unbounded! Not a very fun optimization problem. Given that the AFT model is more resilient to missing covariates and more interpretable, there's a strong argument to use AFT, even though there is no semi-parametric model.

One more reason to favor parametric models over semi-parametric is that they can be easier to generalize. For example, if one wants to perform a Bayesian analysis, it's much easier with a parametric model. Or if one wants to build a cure-rate model, this is non-identifiable for a semi-parametric model, but is identifiable for a parametric model.