Backward selection for Cox model using R

Question

I want to perform an exploratory Cox regression analysis of medical data using R. I am practicing using the pbc data from the survival function.

Would you recommend performing a backward selection multivariate analysis? Are there any summary data / tables I should create for covariates before modelling? Are there any model diagnostics I should perform? And what would be the consequence of doing this?

I would be very grateful for your help and examples using R; also easy to understand literature recommendations (paper, book, and so on) would be nice.

---

To renew my former question: I understand that a stepwise backward regression will lead to inflated coefficients, deflated p-values, and inflated model fit statistics. However, this approach is very common in medical reports. Would it be possible to draw the conclusion that a covariate is independently associated with an outcome, irrespective the above mentioned drawbacks? And when yes, how reliable would it be?

And again being a Little afraid to ask this what would be the best way in R to perform such an analysis?

Answer 1

I would recommend not performing stepwise model building, unless you are looking for biased (inflated) coefficients, biased (deflated) p-values, and inflated model fit statistics.

The fundamental problem is that all of the inferences in one's final model carry a typically invisible/silent and usually uninterpretable series of "conditional upon all these other choices based on other variables in some order" statements.

References

Babyak, M. A. (2004). What you see may not be what you get: A brief, nontechnical introduction to overfitting in regression-type models. Psychosomatic Medicine, 66:411–421.

Henderson, D. A. and Denison, D. R. (1989). Stepwise regression in social and psychological research. Psychological Reports, 64:251–257.

Huberty, C. J. (1989). Problems with stepwise methods—better alternatives. Advances in Social Science Methodology, 1:43–70.

Hurvich, C. M. and Tsai, C.-L. (1990). The impact of model selection on inference in linear regression. The American Statistician, 44(3):214–217.

Lovell, M. C. (1983). Data mining. The Review of Economics and Statistics, 65(1):1–12.

Malek, M. H. and Coburn, D. E. B. J. W. (2007). On the inappropriateness of stepwise regression analysis for model building and testing. European Journal of Applied Physiology, 101(2):263–264.

McIntyre, S. H., Montgomery, D. B., Srinivasan, V., and Weitz, B. A. (1983). Evaluating the statistical significance of models developed by stepwise regression. Journal of Marketing Research, 20(1):1–11.

Pope, P. T. and Webster, J. T. (1972). The use of an $F$-statistic in stepwise regression procedures. Technometrics, 14(2):327–340.

Rencher, A. C. and Pun, F. C. (1980). Inflation of R$^2$ in best subset regression. Technometrics, 22(1):49–53.

Romano, J. P. and Wolf, M. (2005). Stepwise multiple testing as formalized data snooping. Econometrica, 73(4):1237–1282.

Sribney, B., Harrell, F., and Conroy, R. (2011). Problems with stepwise regression.

Steyerberg, E. W., Eijkemans, M. J., and Habbema, J. D. F. (1999). Stepwise selection in small data sets: a simulation study of bias in logistic regression analysis. Journal of clinical epidemiology, 52(10):935–942.

Thompson, B. (1995). Stepwise regression and stepwise discriminant analysis need not apply here: A guidelines editorial. Educational and Psychological Measurement, 55(4):525–534.

Wilkinson, L. (1979). Tests of significance in stepwise regression. Psychological Bulletin, 86(1):168–174.

Answer 2

Stepwise backward regression may be commonly used, but that doesn't avoid the problems noted by @Alexis. If you can't trust the p-values, you won't be able to judge the statistical reliability of any variable's relation to outcome. And always be cautious about using results of multiple regression to conclude that a covariate is "independently" associated with outcome. Higher-quality clinical journals are now more frequently requiring better statistical analyses, often having a separate review by a statistician, a trend that is to be encouraged.

The R package rms was designed for dealing with these types of regressions in statistically reliable ways. Frank Harrell, the author of the package, has a book "Regression Modeling Strategies" and resources available on-line to guide you through its use.

It is true that the stepAIC function in the MASS package allows for simple stepwise selection, but please don't perpetuate the problems that come from publishing results of such un-validated, over-fit models in the clinical literature. Learning how to do it right from the beginning is preferable, even if you don't get your first (unreliable) "result" so quickly. (I am speaking here from less-than-pleasant personal experience.)

Although you are not a statistician, if you are getting involved in clinical research you have to learn enough about statistics to understand the strengths and weaknesses of different approaches, and so that you can consult intelligently with professional statisticians as needed. As you are still in training, it's very appropriate to ask a statistician at your institution for guidance. Starting that learning now will have a high payoff later in your career.