1
$\begingroup$

I am running a logistic regression to predict Yes/No. I have more than 200 independent variables. I have tried to input all the variables, the result is terrible. It is obvious that one variable will effect to the other variable. 1. Is there any way to reduce the variables? 2. How do I know which variables are important and which are not important? 3. I know there is a way called stepwise. But it seems doesn't work for me. Thank you

$\endgroup$
1
$\begingroup$

There are many approaches to the very general problem of attribute selection you're facing here. However, the two methods me and my fellow consultants have successfully applied many times in real-life projects involving logistic regression are:

1) Predictor ranking. We investigate relationships between every single predictor and the target variable, using dependency measures such as univariate logistic regression , Cramer's V or Information Value. After this preliminary step, we keep only a selected number of highest-ranking variables (e.g. having a highest IV score or lowest p-value in logistic model). One may argue that we do not account for possible interactions that way, but in practice that approach worked pretty well for me on large medical and credit scoring datasets.

2) Representative selection. Peform a principal component or factor analysis on the whole set of independent variables to find groups of highly correlated ones. From each such group, pick only 1 or 2 variables, thus reducing the total number. Again, there are many fields where typically you have several related variables and therefore this approach provides good results.

Hope some of these suggestions will prove useful.

$\endgroup$
  • $\begingroup$ Hi Jacek, thank you so much for your ideas. I don't know univariate logistic regression , Cramer's V or Information Value before, but I will have some try. If I want to use R or Python to do this, could you please let me know which package should I use? $\endgroup$ – qqqwww Jul 19 '16 at 22:57
  • $\begingroup$ You can do logistic regression using glm method in R. The Cramer V and Information Value measures can be foudn in 'lsr' and 'woe' packages, respectively. $\endgroup$ – Jacek Podlewski Jul 19 '16 at 22:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.