1
$\begingroup$

My objective is to classify credit applicants into goods and bads. I calculated the information value of each feature as my primary dimension reduction technique.

I was concerned to see that some features that are typically very useful in this kind of problem had very low IVs (for example, the max overdue days of a person's credits). Thus, I ran two logistic regressions to see what would happen:

  1. One with the features with an IV $\geq$ 0.02
  2. One with the same features as the previous model plus the ones that are typically used in this sort of problem but had uncommonly low IVs

I was surprised to see that the features that had very low information values are statistically significant at 99% confidence and have relatively large coefficients.

My question is: why does this happen? Is this common?

$\endgroup$
3
  • $\begingroup$ How do you define/calculate information value of each feature? Is that based only on that feature (and the outcome)? $\endgroup$ Apr 21 '20 at 1:41
  • 1
    $\begingroup$ Yes, you isolate the feature and the target to calculate an IV. In general, an information value is a metric that summarizes how a feature accumulates bads and goods. I split a given feature into deciles to calculate the percentage of 0s and 1s in each bin, calculated the weight of evidence and got the IV. I followed this article. $\endgroup$
    – Arturo Sbr
    Apr 21 '20 at 1:45
  • $\begingroup$ See this list of Qs with answers $\endgroup$ Apr 23 '20 at 4:30
3
$\begingroup$

So you are doing variable selection in logistic regression, by one particular version of univariate screening. Such methods should be avoided, see for instance the answer by F Harrell to Model building and selection using Hosmer et al. 2013. Applied Logistic Regression in R. It is well known (and asked&answered to many times on this site, do a search!) that variables deemed uninformative by such methods can in reality be very informative (and vice versa ... .) This has nothing in particular to do with logistic regression per se ... see for instance Why does an insignificant regressor become significant if I add some significant dummy variables?, How can adding a 2nd IV make the 1st IV significant?, Should a predictor, significant on its own but not with other predictors, be included in an overall multinomial logistic regression?, X and Y are not correlated, but X is significant predictor of Y in multiple regression. What does it mean? and toooo many others!

In short, you should find a better way of model building. The method of weight of evidence and information value you link to seem ad-hoc and very aged. Some specific problems is

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.