2
$\begingroup$

I want to to calculate the area of the differences between the (red) null and (blue) empirical distributions of x, like in

this

, the arrow is the KS metric, but is there a name for the area between the curves? And how does it relate with Information Entropy (Shannon)?

If the distributions are not exactly identical, the area will be >0. Just to be clear, I am not interested by Area(red)-Area(blue), but by the integral |red-blue|. Not the difference of integrals, but the integral of absolute differences. The way I see it relating to entropy and information content is when you consider uniform distribution for the null.

Many thanks in advance,

Seb

$\endgroup$
5
  • 2
    $\begingroup$ I don't know what that area is called, but it certainly is not a good metric for how different the null and empirical distribution are. The area could be 0 while both distributions are still very different. Note also that the difference in area is simply the difference in the expected value of both distributions. $\endgroup$ Commented Jan 8, 2018 at 13:13
  • $\begingroup$ @StijnDeVuyst That is an accurate and (seemingly) complete reply. (Well, I suppose you could also point out the obvious: namely, that the difference in expectations has nothing at all to do with entropy.) If you get the time, please consider posting it as an answer. $\endgroup$
    – whuber
    Commented Jan 8, 2018 at 14:24
  • 3
    $\begingroup$ Thanks, there might be a misunderstanding then. If the distributions are not exactly identical, the area will be >0. Just to be clear, I am not interested by Area(red)-Area(blue), but by the integral |red-blue|. Not the difference of integrals, but the integral of absolute differences. The way I see it relating to entropy and information content is when you consider uniform distribution for the null. $\endgroup$
    – SebDL
    Commented Jan 8, 2018 at 14:31
  • $\begingroup$ You should make this clarifications as an edit to the post! Comments are easily overlooked, and can be deleted. $\endgroup$ Commented Jul 8, 2022 at 3:18
  • 1
    $\begingroup$ The signed area will equal the difference in expected values. The integral of the absolute difference is, by definition, the $L^1$ distance between the two distribution functions. $\endgroup$
    – whuber
    Commented Aug 3, 2022 at 14:42

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.