Multi-View Survival Analysis

Question

I have a data set containing various subsets of medical data about a cohort of patients. For example there are blood test results, demographics, medical examination results and a medical history among other things.

I have created survival models on each subset of the data individually using various methods such as CoxPH, Random Survival Forests, CoxBoost and penalized Cox Regression. I am getting quite good values for the concordance index in the data subsets, but I would like to build a model that combines all of the individual models, in an attempt to improve accuracy.

One option is to select the most important features from the individual data subsets and combine those and build a survival model from those. This does improve accuracy. But is there a way to combine the results of the individual survival models themselves, perhaps by simply aggregating their results in some way like majority voting or by having a higher level algorithm make a decision based on the decisions of the individual survival models? Would this be valid in the case of the Cox PH model for example? What are the issues to consider?

Answer 1

It is possible to combine the prediction from several survival models by using techniques from ensemble learning. In fact, a Random Survival Forest is already an ensemble of survival trees whose predictions are aggregated to form the final prediction of the entire forest. You can use a similar approach when combining predictions from different survival models, which would be called an heterogeneous ensemble.

If you have models that just output a single risk score, which would be the case for all Cox-like models, you just have to aggregate a list of numbers. The simplest way is to just compute an average; or, you can train an additional (linear) survival model on top of the risk scores to learn how to fuse individual predictions – this is called stacking. Even more sophisticated you could consider pruning models that either perform poorly or whose predictions are strongly correlated – this is called ensemble selection or ensemble pruning. For an example for the latter, you can checkout the paper Heterogeneous ensembles for predicting survival of metastatic, castrate-resistant prostate cancer patients .

The most important consideration for an ensemble to be better than its individual members is that their predictions are uncorrelated. If predictions are strongly correlated, all models would essentially give you the same answer, thus, aggregation would not give a different answer. In fact, uncorrelated predictions are usually more important than the prediction performance of individual models, as long as individual models perform better than random (concordance index > 0.5).

Answer 2

Your concern about having too many predictors for the number of cases is valid, but there are better ways to proceed than trying to combine models based on separate subsets of predictors within the same cohort.

The problem is that survival models have an inherent omitted-variable bias. Unlike linear regression, where this is only a problem if the predictors omitted are correlated with those included, omitting any predictor associated with outcome from a survival or logistic regression model tends to bias the estimates of all the included predictor coefficients toward lower magnitudes. This answer has a nice analytical explanation for a conceptually similar situation with probit models.

So the problem you face is trading off that omitted-variable bias against the danger of overfitting with too many predictors.

Say that you have 450 events among your 873 patients. Then you could reasonably try to include 30 or 40 predictors in a standard unpenalized survival model without overfitting. So one approach would be to use clinical judgment to identify the 30 or 40 predictors among all the data types that are most likely to be related to outcome, and use them in a standard Cox regression.

But you already are using penalized approaches in some of your modeling attempts. Your "penalized Cox regression" is presumably a ridge regression or LASSO, which cuts down the effective number of predictors and adjusts the coefficient values to reduce overfitting. A ridge regression using all of the predictors might work surprisingly well; there's no reason to do ridge on separate subsets of them.

Random forests also tend to minimize the possibility of overfitting as they don't use all of the predictors at each decision point. Boosted trees don't typically overfit unless you use too many trees. These tree-based methods are essentially ensembles of models to start with.

Different types of models have different strengths, and there might be something to be said for combining them. So you could consider the following approach: develop different types of models based on all of the predictors reasonably associated with outcome and then combine the models in some way. For example, Mark van der Laan and colleagues recommend what they call targeted maximum likelihood estimation in which they build large numbers of different types of models and then develop a parametric model combining the models' predictions near a particular point of interest (e.g., survival at 3 years) that is optimized by maximum likelihood techniques. That combination of information among several models can lessen any overfitting provided by any single model.

That said, you could be just as well off in practice with one of a well chosen standard Cox model, ridge regression, or one of the tree-based approaches. Penalized approaches like ridge regression have the advantage that you can easily choose to include some critical predictors without penalization and include all other predictors with penalization.

One important point here is to get away from analyzing separate subsets of predictors and to include together as many predictors as are reasonably related to outcome in the set of predictors that you evaluate for your model building. A second point, critical but not directly raised in your your question, is to evaluate your model building process with methods like bootstrapping to get an estimate of generalizability and bias. This page has an outline and links to further details.