Is IV really invariant to the number of bins? I read several times that "the IV formula is fairly invariant to the number of bins".
This is no very intuitive to me, because:


*

*IV is a sum of positive numbers (one number per bucket). Therefore, if you increase the number of buckets, you increase the number of positive elements in the sum -> you could expect the final number to increase as well.

*You could argue that splitting into more buckets would lead to a decrease in the IV in each bucket. This decrease would counterbalance the effet of point (1). But this totally depends on the distributions of the predictive attribute and of the target variable, and I don't see any reason why this would always be true.
To understand the situation better, I generated two random variables X and Y. I computed their IVs by splitting X into 2 and 5 buckets. I performed the simulation several times, and always got a much higher IV when binning into 5 buckets than into 2.

Therefore, am I missing something or is the statement "the IV formula is fairly invariant to the number of bins" wrong?
 A: Your claim sounds similar to the idea underlying the Hosmer–Lemeshow test. This test has been shown to be sensitive to the choice of groups, and also to the group cutoffs themselves. I've put some details below but I think the answer to your question is "you are right - the IV formula is not invariant to the number of bins"
After looking at your formula for IV, this is basically the average log-likelihood value based on using a logistic regression with 1 categorical/factor variable. To briefly show this, let $n_i$ be the number of observations, $a_i$ be the number of events, and $b_i=n_i-a_i$ be the number of non-events, and $G$ be the number of groups ($G=2,5$ in your examples). Then the logistic regression likelihood is given as
$$L(p_{a1},...,p_{aG})=\sum_i a_i \log(p_{ai})+b_i\log(p_{bi})=\sum_i n_i(f_{ai}-f_{bi})\log\left(\frac{p_{ai}}{p_{bi}}\right)$$
where $f_{ai}=1-f_{bi}$ is the observed proportion of events, with $p_{ai}=1-p_{bi}$ is the probability of an event under the model. The MLE estimates are $\hat{p}_{ai}=f_{ai}=1-\hat{p}_{bi}$. Substituting these MLEs into the likelihood gives
$$L(\hat{p}_{a1},...,\hat{p}_{aG})=\sum_i n_i(f_{ai}-f_{bi})\log\left(\frac{f_{ai}}{f_{bi}}\right)=\sum_i n_i IV_i$$
I think this is probably where the claim of invariant comes from, as IV is a group "average goodness of fit" statistic.
But this clearly does not apply as a universal rule. It requires the relationship to be monotonic. ie proportion of increases/decrease as the predictor group increase - group 5 violates this in your example - you get a "V" shape if you plot % vs group number. I expect you might find a different pattern again if you did a "maximum #bins table" where $G$ is equal to the number of distinct values $x$ takes.
This condition also requires you to be collapsing the groups over the "flat" parts of your data. e.g. if group 1-3 proportions are all equal, then the IV you get will also match when you combine the groups into 1 group.
