My objective is to classify credit applicants into goods and bads. I calculated the information value of each feature as my primary dimension reduction technique.
I was concerned to see that some features that are typically very useful in this kind of problem had very low IVs (for example, the max overdue days of a person's credits). Thus, I ran two logistic regressions to see what would happen:
- One with the features with an IV $\geq$ 0.02
- One with the same features as the previous model plus the ones that are typically used in this sort of problem but had uncommonly low IVs
I was surprised to see that the features that had very low information values are statistically significant at 99% confidence and have relatively large coefficients.
My question is: why does this happen? Is this common?