I'm trying to find a model that will help predict the outcome of the dependent variable (binary). My test subset has 6,000 observations My variables are as follow:

  • 6 categorical (with over 30 levels)
  • 20 continuous
  • 45 continuous (percentage)

I've tried many things for variable selection:

  • PROC Logistic with Stepwise
  • PROC Logistic with Stepwise followed by an all-subset research
  • PROC Hpgenselect with the Stepwise method (select sl, choose SBC or AIC)
  • PROC Hpgenselect with the Lasso method

No matter which method, I rarely get more than 5-6 variables selected, and if I try to score my test subset, I don't get anything better than 55% of correctly classified prediction.

I've also tried by adding modified variables (square, log) and by adding interaction terms.

Is there anything I'm missing?

  • $\begingroup$ Things may not be as bad as they seem. How does this 55% correct classification compare with the proportion based on the actual dependent variable? In other words, if you treat predicted (yes/no) vs actual (yes/no) as a two-way table, how does it breakout into false positives and negatives? $\endgroup$ – Mike Hunter Nov 18 '16 at 2:40
  • $\begingroup$ Around 40% for both false positive and false negatives. $\endgroup$ – YH_90 Nov 19 '16 at 2:40
  • $\begingroup$ Are you able to post the full two-way table? $\endgroup$ – Mike Hunter Nov 19 '16 at 14:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.