1
$\begingroup$

I have used many times in a multiple logistic regression the criteria of p-value=0.25 like pre-filter variable selection using bivariate logistic regression , then I use a MANUAL stepwise (backward) to finish the variable selection (p-value=0.05) (only main effects models).

I wonder if its possible use this method in multiple linear regression (pre-filter variable selection using bivariate linear regression), then... I have looking in literature this topic but only I got for multiple logistic regression (only main effects models).

I know that stepwise variable selection have a lot of critics in literature (I repeat: I use a manual method, not automated method...) but my focus is in bivariate linear regression like pre-filter using p-value=0.25 in variable selection...

Could use correlation analysis like a pre filter?

I would grateful some reference if exist...

Thanks in advance.

Regards, Rodrigo.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ I would not do this. Pre-filtering fails to account for confounding variables, and so your selection method could miss important effects. See this paper for more pubmed.ncbi.nlm.nih.gov/8699212. Finally, stepwise is a method fraught with faults. See Frank Harrell's problems with stepwise regression or these artilces journalofbigdata.springeropen.com/articles/10.1186/… $\endgroup$
    – Demetri Pananos
    Commented Dec 2, 2020 at 22:43
  • $\begingroup$ Only I use manual stepwise and main effects model. Thanks for papers... $\endgroup$
    – Rodrigo_BC
    Commented Dec 2, 2020 at 22:46
  • 1
    $\begingroup$ You're still subject to the criticisms in those papers. Enough evidence exists that these methods do not accomplish what users think they do. Use them at your own peril. $\endgroup$
    – Demetri Pananos
    Commented Dec 2, 2020 at 22:47
  • $\begingroup$ They use: "bivariable analysis is greater than an arbitrary value (often p = 0.05)"... I use p-value= 0.25 in bivariable analysis... I know that stepwise have a lot of critics but main interest is in bivariate regression... $\endgroup$
    – Rodrigo_BC
    Commented Dec 2, 2020 at 22:50
  • 2
    $\begingroup$ The problem exists in the estimation of the effect. The significance doesn't really matter all that much. You're free to try and demonstrate that the cut off threshold of p=0.25 leads to the type of behaviour you expect, but I'm rather confident it will not. $\endgroup$
    – Demetri Pananos
    Commented Dec 2, 2020 at 22:53

1 Answer 1

Reset to default
1
$\begingroup$

Automated variable selection methods don't respect the causal model used to inform model development. These approaches can omit important confounders, include colliders, and inappropriately (depending on your precise question) include mediating variables. Depending on how interactions are handled, important effect modifiers could also be missed. So this is their first problem: the interpretation of each independent variable in a model is contingent on the other variables included/not included in the model and this cannot be both fully automated and expected to be sensible at the same time.

The second problem here is that all of the problems of stepwise/forward selection/backward elimination apply whether a computer implements the algorithm or you do. While a manual approach might, in theory, give opportunities to consider individual decisions, if the process is objectively performed according to pre-specified criteria, the only difference between the manual and computer-performed version will be that the former will take more time to do. A pre-screening approach is just a single round of forward selection and so will still overestimate the magnitude of effects and underestimate their p-values.

Note that iterative decisions based on information criteria (e.g. AIC and BIC) have the same problems. I strongly recommend reading Frank Harrell's Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis as this contains a lot of practical wisdom and the results of many, many simulations that Frank's already done on our behalf.

The only reasons for using any of these approaches, in my opinion, are if you are not concerned about generalisability (you simply want to fit the simplest adequate model to a set of data) or to identify a set of variables for future investigation. If you want to make generalisable conclusions based on the data in front of you, don't use any form of automated variable selection.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks but I dont use an automated method. I wrote that use a manual stepwise starting with a full model (backward method)... $\endgroup$
    – Rodrigo_BC
    Commented Dec 3, 2020 at 0:01
  • 2
    $\begingroup$ When I say "automated", I'm not referring to who/what does the work, but rather to the data-driven nature of the process (rather than an expert-driven selection of the relevant variables before looking at the data). It doesn't matter whether you do the modelling and variable selection work by hand or the computer does it for you behind the scenes, the final model selected and so the end result is identical, including the biases for your point estimates and p-values. $\endgroup$
    – user215517
    Commented Dec 3, 2020 at 4:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.