# Stepwise logistic regression

I am working with a dataset of 1000 individuals, 200 of which are disease positive. I have run a logistic regression with 25 predictors to identify overall which variables are significantly predictive. Straightforward...

However, I also want to identify which variables account for the greatest amount of variability for males vs. females, and see if there are differences in which variables pop. I considered modeling gender x predictor interaction terms, but that essentially doubles my number of predictors. I proceeded with a forward logistic regression and what I noticed was that by the last iteration, the model correctly identified a high percentage of non-disease group (>95%) but was very poor in correctly identifying the disease group. If anything, I would prefer a false-positive model (for clinical reasons)!

So I played around and took a random sample of 200 from the non-disease group and ran analyses with those individuals and found that the final iteration of the forward LR correctly predicted a high percentage of both groups. Therefore it seemed that using the whole sample yielded a model biased toward the larger group.

In reading through these pages and other sources, it seems that sub-sampling isn't viewed positively regarding LR, but I could not find anything about using it in an iterative, stepwise procedure.

So my questions are:

1) Is sub-sampling acceptable for a stepwise LR with such a disparate proportion of dichotomous variable?

2) If not, what other procedure(s) should I consider? (e.g., exact logistic regression?)

• What's your question? – Jeremy Miles Mar 9 '15 at 15:59
• Added questions for clarity. My apologies. – Bill Black Mar 9 '15 at 16:07
• I don't know but came up with a lot more questions. (1) 25 variables with 200 events --> 8 EPV. Is this an issue? (2) Are you interested in prediction or characterizing interaction? (choose one) (3) Are there issues with forward selection? (4) So I played around and took a random sample of 200 from the non-disease group pharagraph does not make sense. (5) What do you mean by subsampling? (6) How should one evaluate a logistic regression model? – charles Mar 9 '15 at 16:27
• " ... the model correctly identified a high percentage of non-disease group (>95%) but was very poor in correctly identifying the disease group. ... " How did you reach this conclusion? By making some hard classification, based on some cutoff? That is the wrong way to go about, it introduces a non-proper scoring rule! Search this site for "scoring rule". So that's the wrong way to evaluate the model! So tell us how you reached that conclusion. – kjetil b halvorsen Mar 9 '15 at 16:40
• Charles: 1) Overall 1000 participants, but yes 200 of those were "events." 2) Both really. First, determining the significant predictors for the overall sample and Second, characterization of the interaction (Male/Female). 3)I'm not aware of any issues with forward selection 4) Sorry - I have 800 people w/o disease and 200 with. So I took a random sample of 200/800 to match the sample size of those WITH the disease 5) See #4. 6) We are evaluating based on the percentage of correct classification and Nagelkerke's Rsq – Bill Black Mar 9 '15 at 16:48