6
$\begingroup$

I am building a Cox model containing around 8 variables. Two of the variables that are different measures of the same thing. Consequently, they are correlated with each other. When included in separate models, both show a strong association with survival.

I have read conflicting opinion regarding inclusion of correlated variables within the same model. Is it acceptable to use correlated variables in the same model?

When they are used in the same model, one variable remains significantly associated with outcome, while the association for the second loses significance. Does this tell me anything about these variables? Is the former variable a better predictor of survival than the second?

Many thanks for any advice.

$\endgroup$
2
  • 1
    $\begingroup$ Could you please say a bit more about how you are planning to use your Cox model after it is built? $\endgroup$ – EdM Nov 30 '16 at 21:46
  • 1
    $\begingroup$ My main interest is comparing the two variables and therefore the only information I want from the model is which of the two variables is a stronger predictor of survival. $\endgroup$ – dhmallon Nov 30 '16 at 23:23
8
$\begingroup$

There is no rule against including correlated predictors in a Cox or a standard multiple regression. In practice it is almost inevitable, particularly in clinical work where there can be multiple standard measures of the severity of disease.

Determining which of 2 "measures of the same thing" is better, however, is difficult. When you have 2 predictors essentially measuring the same thing, the particular predictor that seems to work the best may depend heavily on the particular sample of data that you have on hand.

Bootstrapping as suggested by @smndpln can help show the difficulty. If you run a model including both predictors on multiple bootstrap samples, you might well find that only 1 of the 2 is "significant" in any one bootstrap, but the particular predictor found "significant" is likely to vary from bootstrap to bootstrap. This is an inherent problem with highly correlated predictors, whether in Cox regression or standard multiple regression.

You could try LASSO to see whether either or both of the predictors is maintained in a final model that minimizes cross-validation error, but the particular predictor maintained is also likely to vary among bootstrap sample.

You could try comparing nested models. Run the Cox regression first with the standard predictor, then see whether adding your novel predictor adds significant information with anova() in R or a similar function in other software. Then reverse the order, starting with your novel predictor and seeing whether adding the standard predictor adds anything. But if the 2 predictors are highly correlated, it's unlikely that either will add to what's already provided by the other.

You could also compare the 2 models differing only in which of the 2 predictors is included with the Akaike Information Criterion (AIC). This can show which model is "better" on a particular sample. There are, however, no statistical tests to show how big a difference in AIC is "significant." I suppose you could do this comparison on multiple bootstrap samples to get some measure of "significance," but even then you may be unlikely to find a "significant" difference unless your novel predictor is substantially better than the standard predictor that already measures the same thing. And I would worry about whether any differences you find would necessarily hold in other data samples.

Finally, you might consider proposing a model that includes both measures of the phenomenon in question. For prediction, your model need not be restricted to independent variables that are "significant" by some arbitrary test (unless you have so many predictors that you are in danger of over-fitting). Or you could use ridge regression, which can handle correlated predictors fairly well and minimizes the danger of over-fitting.

$\endgroup$
4
  • 1
    $\begingroup$ One could add that sometimes non-statistical considerations help too. One might choose the one which is cheapest or least invasive to collect. $\endgroup$ – mdewey Dec 1 '16 at 10:13
  • $\begingroup$ In relation to your final paragraph where you propose inclusion of both measures - clinically this would certainly make sense. However, when I include both, one remains highly significant, while the other loses significance. Can be taken from this? Is it possible to say that this is evidence that the variable that maintains significance is a better predictor than the variable that loses signifciance? $\endgroup$ – dhmallon Dec 5 '16 at 14:54
  • 1
    $\begingroup$ Such issues of nominal "significance" when variables are correlated are common; see here for example. My guess is that if you repeated your analysis on multiple bootstrap samples of your data, you would sometimes find one and sometimes the other "significant." Ridge regression provides a coherent way to combine correlated predictors in a model without over-fitting. Note that unless you have about 15*8=120 events in your Cox model, you may already be over-fitting a standard Cox regression, based on the rule of thumb of 15 events per predictor. $\endgroup$ – EdM Dec 5 '16 at 16:20
  • $\begingroup$ For anyone else who has the same problem, I found the ridge() method to be the most robust. model <- coxph(Surv(survival_time, censored) ~ ridge(a,b,c,d,e,f,g, theta=5), data=dataframe) print(exp(model$coefficients)) $\endgroup$ – dhmallon Dec 7 '16 at 22:50
1
$\begingroup$

The usage of correlated predictors in a model is called colinearity, and is not something that you want. You need to use a dimensionality reduction approach.

The simplest way to avoid multicolinearity is to perform a principal component analysis (PCA) from the two correlated variables. If the correlation is high, as you are suggesting, then the first component will explain a really high portion of variance. Then, you can use the first component in your subsequent analysis. ( to reduce data dimensionality, you can use MatLab functions pca() or princomp(), and then use biplot to visualize your principal components.)

About comparing the two predictors, an accepted approach seems to involve the usage of bootstrap to generate a distribution of correlations for each predictor. Then you can measure the difference between the two distributions with an effect size metric (like Cohens' d). Function bootstrp() is what you need.

Hope this helps.

$\endgroup$
2
  • $\begingroup$ Thanks for this. Importantly, the two variables mentioned are different scores, one of which is novel, the other is the conventional score. My ultimate goal is to assess which one is a better predictor - something PCA would prevent. $\endgroup$ – dhmallon Nov 30 '16 at 23:19
  • $\begingroup$ I updated the answer $\endgroup$ – smndpln Nov 30 '16 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.