3 roundtrip intuition

# Intuition

Kullback-Leibler Divergence can be interpreted to mean

how many bits of information we expect to lose is we use $$Q$$ instead of $$P$$.

Thus the Population Stability Index is the "roundtrip loss":

how many bits of information we expect to lose is we use $$Q$$ instead of $$P$$ and then use that again to go back to $$Q$$.

# Values

It appears that the Population Stability Index is closely related to the G-test:

$$\mathrm{PSI}(P,Q) = \frac{G(P,Q) + G(Q,P)}{2N}$$

(and thus can be computed using scipy.stats.power_divergence, as well as directly).

Therefore the p-values corresponding to PSI can be computed using the $$\chi^2$$ distribution:

import scipy.stats as st
print "            ","     ".join("DF=%d" % (df) for df in [1,2,3])
for psi in [0.1, 0.25]:
print "PSI=%.2f  %s" % (psi, "".join(
" %5f" % (st.distributions.chi2.sf(psi,df)) for df in [1,2,3]))

DF=1       DF=2       DF=3
PSI=0.10     0.751830   0.951229   0.991837
PSI=0.25     0.617075   0.882497   0.969140


Here PSI is the Population Stability Index and DF is the number of degrees of freedom (DF=n-1 where n is the number of distinct values that the variable takes).

Interestingly enough, the official "interpretation" of the PSI value completely ignores DF.

It appears that the Population Stability Index is closely related to the G-test:

$$\mathrm{PSI}(P,Q) = \frac{G(P,Q) + G(Q,P)}{2N}$$

(and thus can be computed using scipy.stats.power_divergence, as well as directly).

Therefore the p-values corresponding to PSI can be computed using the $$\chi^2$$ distribution:

import scipy.stats as st
print "            ","     ".join("DF=%d" % (df) for df in [1,2,3])
for psi in [0.1, 0.25]:
print "PSI=%.2f  %s" % (psi, "".join(
" %5f" % (st.distributions.chi2.sf(psi,df)) for df in [1,2,3]))

DF=1       DF=2       DF=3
PSI=0.10     0.751830   0.951229   0.991837
PSI=0.25     0.617075   0.882497   0.969140


Here PSI is the Population Stability Index and DF is the number of degrees of freedom (DF=n-1 where n is the number of distinct values that the variable takes).

Interestingly enough, the official "interpretation" of the PSI value completely ignores DF.

# Intuition

Kullback-Leibler Divergence can be interpreted to mean

how many bits of information we expect to lose is we use $$Q$$ instead of $$P$$.

Thus the Population Stability Index is the "roundtrip loss":

how many bits of information we expect to lose is we use $$Q$$ instead of $$P$$ and then use that again to go back to $$Q$$.

# Values

It appears that the Population Stability Index is closely related to the G-test:

$$\mathrm{PSI}(P,Q) = \frac{G(P,Q) + G(Q,P)}{2N}$$

(and thus can be computed using scipy.stats.power_divergence, as well as directly).

Therefore the p-values corresponding to PSI can be computed using the $$\chi^2$$ distribution:

import scipy.stats as st
print "            ","     ".join("DF=%d" % (df) for df in [1,2,3])
for psi in [0.1, 0.25]:
print "PSI=%.2f  %s" % (psi, "".join(
" %5f" % (st.distributions.chi2.sf(psi,df)) for df in [1,2,3]))

DF=1       DF=2       DF=3
PSI=0.10     0.751830   0.951229   0.991837
PSI=0.25     0.617075   0.882497   0.969140


Here PSI is the Population Stability Index and DF is the number of degrees of freedom (DF=n-1 where n is the number of distinct values that the variable takes).

Interestingly enough, the official "interpretation" of the PSI value completely ignores DF.

2 sample size

It appears that the Population Stability Index is closely related to the G-test:

$$\mathrm{PSI}(P,Q) = \frac{G(P,Q) + G(Q,P)}{2}$$$$\mathrm{PSI}(P,Q) = \frac{G(P,Q) + G(Q,P)}{2N}$$

(and thus can be computed using scipy.stats.power_divergence, as well as directly).

Therefore the p-values corresponding to PSI can be computed using the $$\chi^2$$ distribution:

import scipy.stats as st
print "            ","     ".join("DF=%d" % (df) for df in [1,2,3])
for psi in [0.1, 0.25]:
print "PSI=%.2f  %s" % (psi, "".join(
" %5f" % (st.distributions.chi2.sf(psi,df)) for df in [1,2,3]))

DF=1       DF=2       DF=3
PSI=0.10     0.751830   0.951229   0.991837
PSI=0.25     0.617075   0.882497   0.969140


Here PSI is the Population Stability Index and DF is the number of degrees of freedom (DF=n-1 where n is the number of distinct values that the variable takes).

Interestingly enough, the official "interpretation" of the PSI value completely ignores DF.

It appears that the Population Stability Index is closely related to the G-test:

$$\mathrm{PSI}(P,Q) = \frac{G(P,Q) + G(Q,P)}{2}$$

(and thus can be computed using scipy.stats.power_divergence, as well as directly).

Therefore the p-values corresponding to PSI can be computed using the $$\chi^2$$ distribution:

import scipy.stats as st
print "            ","     ".join("DF=%d" % (df) for df in [1,2,3])
for psi in [0.1, 0.25]:
print "PSI=%.2f  %s" % (psi, "".join(
" %5f" % (st.distributions.chi2.sf(psi,df)) for df in [1,2,3]))

DF=1       DF=2       DF=3
PSI=0.10     0.751830   0.951229   0.991837
PSI=0.25     0.617075   0.882497   0.969140


Here PSI is the Population Stability Index and DF is the number of degrees of freedom (DF=n-1 where n is the number of distinct values that the variable takes).

Interestingly enough, the official "interpretation" of the PSI value completely ignores DF.

It appears that the Population Stability Index is closely related to the G-test:

$$\mathrm{PSI}(P,Q) = \frac{G(P,Q) + G(Q,P)}{2N}$$

(and thus can be computed using scipy.stats.power_divergence, as well as directly).

Therefore the p-values corresponding to PSI can be computed using the $$\chi^2$$ distribution:

import scipy.stats as st
print "            ","     ".join("DF=%d" % (df) for df in [1,2,3])
for psi in [0.1, 0.25]:
print "PSI=%.2f  %s" % (psi, "".join(
" %5f" % (st.distributions.chi2.sf(psi,df)) for df in [1,2,3]))

DF=1       DF=2       DF=3
PSI=0.10     0.751830   0.951229   0.991837
PSI=0.25     0.617075   0.882497   0.969140


Here PSI is the Population Stability Index and DF is the number of degrees of freedom (DF=n-1 where n is the number of distinct values that the variable takes).

Interestingly enough, the official "interpretation" of the PSI value completely ignores DF.

1

It appears that the Population Stability Index is closely related to the G-test:

$$\mathrm{PSI}(P,Q) = \frac{G(P,Q) + G(Q,P)}{2}$$

(and thus can be computed using scipy.stats.power_divergence, as well as directly).

Therefore the p-values corresponding to PSI can be computed using the $$\chi^2$$ distribution:

import scipy.stats as st
print "            ","     ".join("DF=%d" % (df) for df in [1,2,3])
for psi in [0.1, 0.25]:
print "PSI=%.2f  %s" % (psi, "".join(
" %5f" % (st.distributions.chi2.sf(psi,df)) for df in [1,2,3]))

DF=1       DF=2       DF=3
PSI=0.10     0.751830   0.951229   0.991837
PSI=0.25     0.617075   0.882497   0.969140


Here PSI is the Population Stability Index and DF is the number of degrees of freedom (DF=n-1 where n is the number of distinct values that the variable takes).

Interestingly enough, the official "interpretation" of the PSI value completely ignores DF.