Information Value Definition on Siddiqui "Credit Risk Scorecards" Book

Question

I was studying about information value, and saw the definition on "Siddiqui, Credit Risk Scorecards". I'm searchin for some references on why the formula there is that way, and also for the thresholds he assumes for variable selection, can someone help me? At first I assumed that information value was almost the same as mutual information, but the formulas don't match ( i tried both pointwise mututal information and expected mutual information as described here:Information gain and mutual information: different or equal?). The definition on the book is:

$$ \sum_i (DistrGood_i - DistrBad_i) \times \ln\frac{DistrGood_i}{DistrBad_i} $$

Answer 1

Mutual Information (sometimes called Information Gain) is a decrease in information entropy of one random variable when another is known. It measures dependency (not only linear) between random variables. It has formula: $$ I(X,Y) = \sum\limits_{x, y}P(X=x, Y=y)\ln\left(\frac{P(X=x,Y=y)}{P(X=x)P(Y=y)}\right) $$ It can be used as feature utility metric.

Information Value measures predictive power of a feature and is given by formula (sometimes modified with binning): $$ IV(X, Y) = \sum_{x}(P(X=x|Y=1) - P(X=x|Y=0))\ln\left(\frac{P(X=x|Y=1)}{P(X=x|Y=1)}\right) $$

Although similar, Information Value and Information Gain (Mutual Information) are different things. Both are used in univariate feature selection. Both can be expressed in terms of Kullback-Leibler divergence (for calculation for IV check this answer).