1
$\begingroup$

I had set of binary variables. To apply logistic regression, I have checked association between dependent and independent variables and considered only those independent variables in the model which came to be associated with dependent variable. My query is whether it is an appropriate way of fitting logistic regression model.

$\endgroup$
  • $\begingroup$ yes: everyone does it. no: still subject to limitations seen in stepwise regression. this isn't equivalent to using a full pre-specified model. $\endgroup$ – charles Dec 4 '13 at 16:35
  • 1
    $\begingroup$ No, this is data dredging. It yields too optimistic results, especially for smaller samples. $\endgroup$ – Michael M Dec 4 '13 at 17:31
  • $\begingroup$ @charles: Well not everyone, & those that do are careful to validate the resulting model on a hold-out sample or a new data-set. $\endgroup$ – Scortchi - Reinstate Monica Dec 5 '13 at 11:20
  • $\begingroup$ @Scortchi : So how should I proceed? Actually when I was taking all the variables, I was getting variables with negative coefficients which is not making business sense as independent variables are such a way that increasing independent variable will definitely increase dependent variable. That is why I have checked association between DV and IV and placed considerd only those variables which are associated with DV. $\endgroup$ – Kartikeya Pandey Dec 5 '13 at 13:41
  • $\begingroup$ That's a very broad topic. I'd suggest looking at posts under the model-selection & feature-selection tags, e.g. this, this, & this - as I'm writing this I see a good answer from @charles. $\endgroup$ – Scortchi - Reinstate Monica Dec 5 '13 at 14:49
3
$\begingroup$

(1) Do you really need a smaller model? If not, you're set.
(2) Can you honestly pre-specify your model? From your knowledge of the field can you choose a subset of predictors your interested in without using your knowledge of this dataset? If so, you're set.
(2.5) If all your data valid? Assess this without looking at outcomes.
(3) Consider using some form of shrinkage method. Ridge, lasso, elastic-net... Ameliorates some of the issues with model reduction.
(4) If none of the above apply to you, consider some traditional form of model reduction. Stepwise or the uni-variate screening mentioned in the post. Be aware that with EPV<50 this approach has limitations. Perform some form of internal validation. Bootstrap generally preferable (unless very large.? >10,000 obs if often quoted here, but no consensus that I'm aware of. in smaller datasets split sample unstable, also doesn't give you stability of variable selection provided by bootstrap). In each bootstrap include the whole model building process (univariate select or stepwise).

$\endgroup$
  • $\begingroup$ Under (2) I'd add doing other things to the candidate predictors than simply selecting some from among them (e.g. variable clustering, PCA, transformations). Harrell (2001), Regression Modelling Strategies is a good reference for various data reduction techniques. $\endgroup$ – Scortchi - Reinstate Monica Dec 5 '13 at 15:11
  • $\begingroup$ @Scortchi As far as I know, PCA can not be applied to binary variables.Let me know in case I misunderstood your point or I have some gap in understanding of PCA. $\endgroup$ – Kartikeya Pandey Dec 5 '13 at 16:43
  • $\begingroup$ @Charles In uni-variate screening, I should consider one IV variable at a time. This is how I will get to know variables which need to be taken in final model and then will consider only those variables in full model which came to be significant in uni variate case. $\endgroup$ – Kartikeya Pandey Dec 5 '13 at 16:48
  • 2
    $\begingroup$ That is an accurate description of uni-variate screening. In addition, usually a more generous p-value of 0.2 is used for cut-off for model inclusion than in stepwise. The take home point is that people forget that uni-variate screening has the same limitations as stepwise selection methods. (whether these limitations can be ameliorated by higher the p-value threshold is another issue, and one I have no opinion on). @gung has written a few answers on the limitations of stepwise (and more generally data driven model building) if you search this site. $\endgroup$ – charles Dec 5 '13 at 16:59
  • 2
    $\begingroup$ To be clear, there's no reason to expect univariate screening to work well in general. In addition to the usual defects of model reduction without any compensating shrinkage, you run the risk of omitting important predictors just because they're confounded with others. If you use it, heed @charles' advice & make sure to re-run the screening process in each cross-validation step. $\endgroup$ – Scortchi - Reinstate Monica Dec 5 '13 at 21:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.