How come variables with low information values may be statistically significant in a logistic regression? My objective is to classify credit applicants into goods and bads. I calculated the information value of each feature as my primary dimension reduction technique.
I was concerned to see that some features that are typically very useful in this kind of problem had very low IVs (for example, the max overdue days of a person's credits). Thus, I ran two logistic regressions to see what would happen:


*

*One with the features with an IV $\geq$ 0.02

*One with the same features as the previous model plus the ones that are typically used in this sort of problem but had uncommonly low IVs


I was surprised to see that the features that had very low information values are statistically significant at 99% confidence and have relatively large coefficients.
My question is: why does this happen? Is this common?
 A: So you are doing variable selection in logistic regression, by one particular version of univariate screening.  Such methods should be avoided, see for instance the answer by F Harrell to Model building and selection using Hosmer et al. 2013. Applied Logistic Regression in R.  It is well known (and asked&answered to many times on this site, do a search!) that variables deemed uninformative by such methods can in reality be very informative (and vice versa ... .) This has nothing in particular to do with logistic regression per se ... see for instance Why does an insignificant regressor become significant if I add some significant dummy variables?,  How can adding a 2nd IV make the 1st IV significant?,  Should a predictor, significant on its own but not with other predictors, be included in an overall multinomial logistic regression?,  X and Y are not correlated, but X is significant predictor of Y in multiple regression. What does it mean?   and toooo many others!
In short, you should find a better way of model building. The method of weight of evidence and information value you link to seem ad-hoc and very aged. Some specific problems is


*

*binning of continuous variables, don't do it. See What is the benefit of breaking up a continuous predictor variable?

*univariate screening is bad 
