7
$\begingroup$

I am trying to find predictors for an outcome. I was taught to perform univariate analyses & put significant variables into a multivariate logistic regression model. Then I remove variables one by one based on p values > 0.05 to obtain the final model.

I saw from some papers that there is another approach. Basically, they do not remove any variable from the multivariate model, adjusting for all.

The first appraoch may not adjust for some potential confounders, but you get a model with less variables, all of which are significant. The second approach adjusts for all, which could be quite a long list. Are there any other important advantages or disadvantages that I should be aware of between the two approaches?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ It's sometimes called "univariable screening" & gets a mention here. See Sun et al (1996), "Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis", J Clin Epidemiol., 49, 8. $\endgroup$
    – Scortchi ♦
    Commented Mar 17, 2016 at 10:28
  • $\begingroup$ Thanks for the link to the interesting article. One alternative mentioned in the article was to fit all the variables into a regression model, then apply backward elimination. One issue I recalled from class is that if the variables are many, it would also affect the model. Is that a valid concern as well? $\endgroup$
    – Rey
    Commented Mar 17, 2016 at 12:12
  • $\begingroup$ First get straight why you want to remove any predictors at all if you thought them worth including at the outset. Is your model over-fitting? Probably worth your reading Algorithms for automatic model selection. $\endgroup$
    – Scortchi ♦
    Commented Mar 17, 2016 at 12:27
  • $\begingroup$ Although the outcome has been investigated in other countries, this is done using local data for the first time. Selection of the predictors were made to the best of my judgment, but I am not 100% sure that they could influence the outcome significantly. From the large pool of available data, I included 13-17 variables in the initial univariate logistic regression. $\endgroup$
    – Rey
    Commented Mar 17, 2016 at 12:52

1 Answer 1

Reset to default
9
$\begingroup$

Approaches that naively select model terms based on some p-values or some AIC cut-offs (either in a multivariate model via some kind of stepwise or other selection or by looking at lots of univariate models) lead to extremely problematic fits that may fit the particular dataset well, but will otherwise not be useful. Models constructed in such a fashion tend to wrongly identify variables as relevant that are not (while not identifying truly relevant variables - if we assume the used model is some reasonable approximation to nature, in which some variables are relevant and some are not) and have poor predictive properties on new datasets. Nevertheless such approaches are still often used and one can even occasionally get such work published in some well-respected journals, but are quite thoroughly discredited in the statistical community. There are a lot of more appropriate approaches, e.g. bootstrapping naive model building approaches, cross-validation, random forests, model averaging, variable selection priors etc. that should be used instead.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for the reply. I went through many journal articles, looking at their analyses write-up and they seem to be using some variations of the method using p-values. For my case to identify significant predictors of an outcome, do you have any suggestions on what may be a more easily implemented procedure among those you listed? $\endgroup$
    – Rey
    Commented Mar 17, 2016 at 12:18
  • 5
    $\begingroup$ The fact that journals allow this is more a reflection of their having poor statistical review. This is really bad practice. Don't use bad ideas as your inspiration; use best practices. $\endgroup$
    – Frank Harrell
    Commented Mar 17, 2016 at 12:54
  • $\begingroup$ This is good advice. Like to check whether you have any opinion on my other question: stats.stackexchange.com/questions/202124/… $\endgroup$
    – Rey
    Commented Mar 18, 2016 at 5:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.