I am running a logistic regression to predict Yes/No. I have more than 200 independent variables. I have tried to input all the variables, the result is terrible. It is obvious that one variable will effect to the other variable. 1. Is there any way to reduce the variables? 2. How do I know which variables are important and which are not important? 3. I know there is a way called stepwise. But it seems doesn't work for me. Thank you
There are many approaches to the very general problem of attribute selection you're facing here. However, the two methods me and my fellow consultants have successfully applied many times in real-life projects involving logistic regression are:
1) Predictor ranking. We investigate relationships between every single predictor and the target variable, using dependency measures such as univariate logistic regression , Cramer's V or Information Value. After this preliminary step, we keep only a selected number of highest-ranking variables (e.g. having a highest IV score or lowest p-value in logistic model). One may argue that we do not account for possible interactions that way, but in practice that approach worked pretty well for me on large medical and credit scoring datasets.
2) Representative selection. Peform a principal component or factor analysis on the whole set of independent variables to find groups of highly correlated ones. From each such group, pick only 1 or 2 variables, thus reducing the total number. Again, there are many fields where typically you have several related variables and therefore this approach provides good results.
Hope some of these suggestions will prove useful.