What is the intuition behind the Population Stability Index?

The "Population Stability Index" for two distributions $P$ and $Q$ is defined as the Symmetrised Kullback-Leibler divergence:

$$\mathrm{PSI}(P,Q) = D_{KL}(P||Q) + D_{KL}(Q||P) = \sum_i(P_i-Q_i)\log\frac{P_i}{Q_i}$$

What is the intuition behind this number?

One can always use the intuition for $D_{KL}$ and say that PSI is

the expected number of extra bits required to code samples from $P$ using a code optimized for $Q$ rather than the code optimized for $P$

plus the expected number of extra bits required to code samples from $Q$ using a code optimized for $P$ rather than the code optimized for $Q$,

but this is quite a mouthful.

Quora and UCAnalytics offer this "interpretation":

• PSI < 0.1: Insignificant change (No action required)
• 0.1 < PSI < 0.25: Some minor change (Start worrying)
• 0.25 < PSI: Major shift in population (Need to delve deeper)

what is the basis for this?

• Can you give some "more formal" reference that defines and uses/explains the PSI? Never heard the term Commented Apr 20, 2017 at 20:56
• @kjetilbhalvorsen: I wish I had something better than the 3(three!) links in the text.
– sds
Commented Apr 20, 2017 at 20:59
• This link seems to be useful: support.sas.com/resources/papers/proceedings10/288-2010.pdf Commented Apr 20, 2017 at 21:07
• I haven't checked that but the original source for these commonly used thresholds is: "An introduction to credit scoring" by E.M. Lewis
– phil
Commented Aug 25, 2017 at 14:50
• I would like to add a link to my dissertation that I recently completed. Please comment on the content and I hope it helps to understand PSI better. scholarworks.wmich.edu/dissertations/3208 Commented Jun 14, 2018 at 18:31

Intuition

Kullback-Leibler Divergence can be interpreted to mean

how many bits of information we expect to lose is we use $$Q$$ instead of $$P$$.

Thus the Population Stability Index is the "roundtrip loss":

how many bits of information we expect to lose is we use $$Q$$ instead of $$P$$ and then use that again to go back to $$Q$$.

Values

It appears that the Population Stability Index is closely related to the G-test:

$$\mathrm{PSI}(P,Q) = \frac{G(P,Q) + G(Q,P)}{2N}$$

where $$N$$ is the total number of observations. (and thus can be computed using scipy.stats.power_divergence, as well as directly using scipy.stats.entropy).

Since $$G\sim\chi^2_{N-1}$$ and $$G(P,Q)$$ and $$G(Q,P)$$ are highly correlated, we can assume that $$N\times\mathrm{PSI}\sim\chi^2_{N-1}$$ (if they were independent instead, we would have had $$2N\times\mathrm{PSI}\sim\chi^2_{2N-2}$$).

Therefore the p-values corresponding to PSI can be computed like this:

import scipy.stats as st
import pandas as pd

def psi_p_value_correlated(psi, n):
"chi2 p-value for PSI assuming G(P,Q) and G(Q,P) are highly correlated"
return st.distributions.chi2.sf(n*psi,n-1)

def psi_p_value_independent(psi, n):
"chi2 p-value for PSI assuming G(P,Q) and G(Q,P) are independent"
return st.distributions.chi2.sf(2*n*psi,2*(n-1))

def psi_p_values(p_value, psis=(0.05, 0.1, 0.25, 0.5), ns=(2, 3, 5, 8, 13)):
return pd.DataFrame({"N":f"{N=}", **{f"{PSI=:.2f}":p_value(PSI, N) for PSI in psis}}
for N in ns).set_index("N", verify_integrity=True)

psi_p_values(psi_p_value_correlated)
PSI=0.05  PSI=0.10  PSI=0.25  PSI=0.50
N=2   0.751830  0.654721  0.479500  0.317311
N=3   0.927743  0.860708  0.687289  0.472367
N=5   0.992809  0.973501  0.869800  0.644636
N=8   0.999737  0.997444  0.959840  0.779777
N=13  0.999999  0.999940  0.993507  0.888813

psi_p_values(psi_p_value_independent)
PSI=0.05  PSI=0.10  PSI=0.25  PSI=0.50
N=2   0.904837  0.818731  0.606531  0.367879
N=3   0.989814  0.963064  0.826641  0.557825
N=5   0.999867  0.998248  0.961731  0.757576
N=8   1.000000  0.999979  0.995466  0.889326
N=13  1.000000  1.000000  0.999851  0.966120


This means that PSI=0.1 with N=3 is due to chance with probability 86% (assuming $$G(P,Q)$$ and $$G(Q,P)$$ are highly correlated).

PS. Interestingly enough, the official "interpretation" of the PSI value completely ignores $$N$$.

• PSI can also be computed using scipy.stats.entropy. Like so PSI(A, B) = entropy(A, B) + entropy(B, A). See here: docs.scipy.org/doc/scipy/reference/generated/… Commented Oct 7, 2022 at 22:07
• Can you elaborate a little more on OP's question "what is the basis for this"? Why the associated $p$-values indicate "insignificant/minor/major" changes? (What I saw is, no matter what DF is, no entries are below the conventional 0.05 threshold). Commented Feb 16, 2023 at 5:00