9
$\begingroup$

Population stability index quantifies the change of a distribution of a variable by comparing data samples in two time periods. It is very commonly used to measure shifts in scores.

It is calculated as follows:
1) The sample from base period is discretized. Usually it is partitioned into deciles
2) The sample from target period is discretized using the same intervals as in first step
$PSI = \sum_{i} (A_{i} - B_{i}) \cdot ln(\frac{A_{i}}{B_{i}})$
Where:
$A_{i}$ - share of i-th bin in base period. $B_{i}$ - share of i-th bin in target period.

Question: What should be done when one of the bins from target sample is empty?

$\endgroup$
  • $\begingroup$ I usually join the class with no elements with an adjacent class that has elements. Hope that helps. C. $\endgroup$ – CSands Dec 13 '13 at 12:49
  • $\begingroup$ @CSands: Thanks for answer, but if I understood correctly then in case when all observations from ninth decile shifts to the tenth decile your solution would join ninth and tenth decile and PSI would be 0, where in reality the distribution has changed significantly. Am I right? $\endgroup$ – Tomek Tarczynski Dec 13 '13 at 13:54
  • $\begingroup$ Rather than using bins, is it possible to use density estimates and replace the sum with an integral? $\endgroup$ – dsaxton Mar 22 '16 at 2:53
  • 1
    $\begingroup$ You could probably make a better estimator by adapting the ideas from my answer here: stats.stackexchange.com/questions/211175/… $\endgroup$ – kjetil b halvorsen Apr 20 '17 at 21:00
3
$\begingroup$

I guess you could consider the empty bins as filled with a very small number. This retains the information and avoids division by zero. And, of course, this way you keep the original bins, which is a good thing.

$\endgroup$
  • 4
    $\begingroup$ This recommendation needs some analysis. For instance, how sensitive is the result to the choice of that "very small number"? If it's relatively insensitive, then why not propose taking the limit as that small number approaches zero from above? If it is sensitive, then the answer would effectively be arbitrary. $\endgroup$ – whuber Sep 29 '15 at 15:37
  • $\begingroup$ Agreed. ln(Ai/Bi) shouldn't grow too fast, because of the logarithm, but still the 'small Bi' should be chosen so that it doesn't invalidate the PSI measure. $\endgroup$ – rjeronimo Sep 29 '15 at 21:45
  • $\begingroup$ I don't see how that is mathematically possible. $\endgroup$ – whuber Sep 30 '15 at 13:10
  • $\begingroup$ for example, if you only consider the % in each bin rounded to 2 decimal places, you could use 0.01 instead of zero. $\endgroup$ – rjeronimo Sep 30 '15 at 21:31
  • $\begingroup$ "very small number" could be Machine epsilon $\endgroup$ – sds Jun 1 '16 at 17:37
0
$\begingroup$

Could you skip it? That is, could you understand it as zero?

The division by zero is uniquely and reasonably determined as 1/0=0/0=z/0=0 in the natural extensions of fractions. We have to change our basic ideas for our space and world Division by Zero z/0 = 0 in Euclidean Spaces Hi roshi Michiwaki, Hiroshi Okumura and Saburou Saitoh International Journal of Mathematics and Computation Vol. 28(2017); Issue 1, 2017), 1 -16.   http://www.scirp.org/journal/alamt     http://dx.doi.org/10.4236/alamt.2016.62007 http://www.ijapm.org/show-63-504-1.html http://www.diogenes.bg/ijam/contents/2014-27-2/9/9.pdf http://okmr.yamatoblog.net/division%20by%20zero/announcement%20326-%20the%20divi http://okmr.yamatoblog.net/ Relations of 0 and infinity Hiroshi Okumura, Saburou Saitoh and Tsutomu Matsuura: http://www.e-jikei.org/…/Camera%20ready%20manuscript_JTSS_A… https://sites.google.com/site/sandrapinelas/icddea-2017

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.