1
$\begingroup$

Recently, I was asked to explain in my peer-review article by the reviewer to elaborate on why I employed variables that showed a trend towards significance (p<0.1) as well as statistical significance (p<0.05) as variables to be included in the multivariate Cox proportional-hazards model.

Since I did saw this in several articles, I also included this in my statistical analysis using SPSS package. Beyond using previous articles as an argument, is there any other in-depth explanation for why are we using initial screening in univariate analysis for p<0.1 and afterwards including them in multivariate Cox proportional-hazards model? Any help and referenced explanation would be of much use.

$\endgroup$

1 Answer 1

5
$\begingroup$

Omitting any outcome-associated predictor from a Cox multiple-regression* model runs a risk of omitted-variable bias. Unlike the situation with ordinary least squares discussed in that linked Wikipedia page, a Cox model can show such bias even if the omitted outcome-associated predictor is uncorrelated with the included predictors. In that respect a Cox model is similar to binomial regression.

The Abstract of the paper Effects of Omitting Covariates in Cox's Model for Survival Data by Jean Bretagnolle and Catherine Huber-Carol, Scandinavian Journal of Statistics 15: 125-138 (1988) puts it clearly with respect to the univariable screening strategy that you used:

when there is only one covariate left in the used model ... the effect on the survival of the covariate under study is always underestimated

The problem is actually opposite in direction from what your reviewer apparently said. Requiring a single predictor to show p < 0.05 in univariable screening before including it in a Cox multiple regression is likely to omit important predictors whose association with outcome only becomes apparent when other predictors are taken into account. It also will lead to bias in the estimates of the predictors included in the Cox multiple regression. The Abstract goes on to say:

... in case of several covariates remaining in the studied model we prove that the same result of underestimation holds for each of them at least up to some fixed time $t_0$ which, in practical cases, is reasonably long. The asymptotic bias resulting from such omissions is not negligible...

Univariable screening (although widely used) is an unreliable strategy for any regression modeling. From Frank Harrell's Regression Modeling Strategies, second edition, pp. 71-72:

Many papers claim that there were insufficient data to allow for multivariable modeling, so they did “univariable screening” wherein only “significant” variables (i.e., those that are separately significantly associated with Y ) were entered into the model. This is just a forward stepwise variable selection in which insignificant variables from the first step are not reanalyzed in later steps. Univariable screening is thus even worse than stepwise modeling as it can miss important variables that are only important after adjusting for other variables.

See this page for why forward stepwise selection (and other automated model selection methods) are problematic.

The Regression Modeling Strategies book and the associated online notes explain better strategies that draw on knowledge of the subject matter and that avoid use of the outcome values to choose predictors to include, particularly in Chapter 4. When you use outcome values to choose predictors, you are likely to overfit the model and things like p-values and confidence intervals will be incorrect.

And when you build a multiple regression model:

Don’t remove insignificant effects from the model when tested separately by predictor. (Regression Modeling Strategies, p. 96)

I suppose that a simple reply to the reviewer would be to say that your higher p-value cutoff than 0.05 reduces omitted-variable bias. But you should go beyond that and re-evaluate whether you should be using univariable screening at all as a precursor to multiple regression.


*It's best current practice to reserve the word "multivariate" for models that have multiple outcomes, as distinguished from "univariate" models with a single outcome. See B. Hildago and M. Goodman, "Multivariate or multivariable regression?" Am. J. Public Health 103:39-40 (2013). Regression models with multiple predictor variables are then "multivariable" or "multiple regression" models, and models with a single predictor variable can be called "univariable."

$\endgroup$
2
  • $\begingroup$ This is well elaborated answer, and I much appreciate the effort, I must say. Since I do not want to go into details, but rather take a stand, and provide an answer to help publish this, although high impact journals readily publish results using univariate screening as a legitimate tool in Cox proportional-hazards model. Hence, I will argue that what you said in the last part that it helps reduce omitted-variable bias in my case. $\endgroup$
    – Shawn93
    Jan 9, 2023 at 13:36
  • 1
    $\begingroup$ @Shawn93 the fact that high-impact journals publish such results does not make univariable screening a legitimate tool. Remember that high-impact journals are interested in publishing apparently novel results. Unfortunately, the overfitting likely to result from univariable screening can tend to lead to "novel" results that don't extend well to new data. That said, I appreciate the situation that you are in and understand why you are proceeding this way. $\endgroup$
    – EdM
    Jan 9, 2023 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.