Consider two distinct probability distributions $P(X)$ and $Q(Y)$---defined on the same domain---with (Shannon) entropy of $H(X)$ and $H(Y)$. I am interested to prove that $$ H(X) \leq H(Y) \implies \sum_{x}\sum_{x'} P(x)P(x')|x-x'| \leq \sum_{y}\sum_{y'} Q(y)Q(y')|y-y'| $$

Any answer, comments, or directions would be appreciated.

  • 2
    $\begingroup$ Why do you believe this would be true? What kind of examples have you looked at? $\endgroup$
    – cardinal
    Jun 1 '15 at 17:17

It's not true.

Take two probability distributions with the same entropy (1 bit):

$$P(0) = \tfrac{1}{2}, \quad P(1) = 0, \quad P(1001) = \tfrac{1}{2}$$ and $$Q(0) = \tfrac{1}{2}, \quad Q(1) = \tfrac{1}{2}, \quad Q(1001) = 0.$$

However, the sum

$$\sum_x \sum_{x'} R(x) R(x') \left|x - x'\right|$$

is $250$ and $0.25$, respectively.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.