6
$\begingroup$

The "Population Stability Index" for two distributions $P$ and $Q$ is defined as the Symmetrised Kullback-Leibler divergence:

$$ \mathrm{PSI}(P,Q) = D_{KL}(P||Q) + D_{KL}(Q||P) = \sum_i(P_i-Q_i)\log\frac{P_i}{Q_i} $$

What is the intuition behind this number?

One can always use the intuition for $D_{KL}$ and say that PSI is

the expected number of extra bits required to code samples from $P$ using a code optimized for $Q$ rather than the code optimized for $P$

plus the expected number of extra bits required to code samples from $Q$ using a code optimized for $P$ rather than the code optimized for $Q$,

but this is quite a mouthful.

Quora and UCAnalytics offer this "interpretation":

  • PSI < 0.1: Insignificant change (No action required)
  • 0.1 < PSI < 0.25: Some minor change (Start worrying)
  • 0.25 < PSI: Major shift in population (Need to delve deeper)

what is the basis for this?

$\endgroup$
  • $\begingroup$ Can you give some "more formal" reference that defines and uses/explains the PSI? Never heard the term $\endgroup$ – kjetil b halvorsen Apr 20 '17 at 20:56
  • 1
    $\begingroup$ @kjetilbhalvorsen: I wish I had something better than the 3(three!) links in the text. $\endgroup$ – sds Apr 20 '17 at 20:59
  • $\begingroup$ This link seems to be useful: support.sas.com/resources/papers/proceedings10/288-2010.pdf $\endgroup$ – kjetil b halvorsen Apr 20 '17 at 21:07
  • $\begingroup$ I haven't checked that but the original source for these commonly used thresholds is: "An introduction to credit scoring" by E.M. Lewis $\endgroup$ – phil Aug 25 '17 at 14:50
  • 3
    $\begingroup$ I would like to add a link to my dissertation that I recently completed. Please comment on the content and I hope it helps to understand PSI better. scholarworks.wmich.edu/dissertations/3208 $\endgroup$ – Bilal Yurdakul Jun 14 '18 at 18:31
2
$\begingroup$

Intuition

Kullback-Leibler Divergence can be interpreted to mean

how many bits of information we expect to lose is we use $Q$ instead of $P$.

Thus the Population Stability Index is the "roundtrip loss":

how many bits of information we expect to lose is we use $Q$ instead of $P$ and then use that again to go back to $Q$.

Values

It appears that the Population Stability Index is closely related to the G-test:

$$ \mathrm{PSI}(P,Q) = \frac{G(P,Q) + G(Q,P)}{2N} $$

(and thus can be computed using scipy.stats.power_divergence, as well as directly).

Therefore the p-values corresponding to PSI can be computed using the $\chi^2$ distribution:

import scipy.stats as st
print "            ","     ".join("DF=%d" % (df) for df in [1,2,3])
for psi in [0.1, 0.25]:
    print "PSI=%.2f  %s" % (psi, "".join(
        " %5f" % (st.distributions.chi2.sf(psi,df)) for df in [1,2,3]))

               DF=1       DF=2       DF=3
PSI=0.10     0.751830   0.951229   0.991837
PSI=0.25     0.617075   0.882497   0.969140

Here PSI is the Population Stability Index and DF is the number of degrees of freedom (DF=n-1 where n is the number of distinct values that the variable takes).

Interestingly enough, the official "interpretation" of the PSI value completely ignores DF.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.