I had set of binary variables. To apply logistic regression, I have checked association between dependent and independent variables and considered only those independent variables in the model which came to be associated with dependent variable. My query is whether it is an appropriate way of fitting logistic regression model.
$\begingroup$
$\endgroup$
5
asked Dec 4, 2013 at 16:26
1 Answer
$\begingroup$
$\endgroup$
6
(1) Do you really need a smaller model? If not, you're set.
(2) Can you honestly pre-specify your model? From your knowledge of the field can you choose a subset of predictors your interested in without using your knowledge of this dataset? If so, you're set.
(2.5) If all your data valid? Assess this without looking at outcomes.
(3) Consider using some form of shrinkage method. Ridge, lasso, elastic-net... Ameliorates some of the issues with model reduction.
(4) If none of the above apply to you, consider some traditional form of model reduction. Stepwise or the uni-variate screening mentioned in the post. Be aware that with EPV<50 this approach has limitations. Perform some form of internal validation. Bootstrap generally preferable (unless very large.? >10,000 obs if often quoted here, but no consensus that I'm aware of. in smaller datasets split sample unstable, also doesn't give you stability of variable selection provided by bootstrap). In each bootstrap include the whole model building process (univariate select or stepwise).
-
$\begingroup$ Under (2) I'd add doing other things to the candidate predictors than simply selecting some from among them (e.g. variable clustering, PCA, transformations). Harrell (2001), Regression Modelling Strategies is a good reference for various data reduction techniques. $\endgroup$ Dec 5, 2013 at 15:11
-
$\begingroup$ @Scortchi As far as I know, PCA can not be applied to binary variables.Let me know in case I misunderstood your point or I have some gap in understanding of PCA. $\endgroup$ Dec 5, 2013 at 16:43
-
$\begingroup$ @Charles In uni-variate screening, I should consider one IV variable at a time. This is how I will get to know variables which need to be taken in final model and then will consider only those variables in full model which came to be significant in uni variate case. $\endgroup$ Dec 5, 2013 at 16:48
-
2$\begingroup$ That is an accurate description of uni-variate screening. In addition, usually a more generous p-value of 0.2 is used for cut-off for model inclusion than in stepwise. The take home point is that people forget that uni-variate screening has the same limitations as stepwise selection methods. (whether these limitations can be ameliorated by higher the p-value threshold is another issue, and one I have no opinion on). @gung has written a few answers on the limitations of stepwise (and more generally data driven model building) if you search this site. $\endgroup$– charlesDec 5, 2013 at 16:59
-
2$\begingroup$ To be clear, there's no reason to expect univariate screening to work well in general. In addition to the usual defects of model reduction without any compensating shrinkage, you run the risk of omitting important predictors just because they're confounded with others. If you use it, heed @charles' advice & make sure to re-run the screening process in each cross-validation step. $\endgroup$ Dec 5, 2013 at 21:37
model-selection&feature-selectiontags, e.g. this, this, & this - as I'm writing this I see a good answer from @charles. $\endgroup$