I know that Jensen-Shannon divergence between two distribution $P$ and $Q$ can be written as follow:

$JS(P||Q) = H(\frac{P+Q}{2}) - \frac{H(P)}{2} - \frac{H(Q)}{2}$

But is there any intuition behind this definition of JS divergence? When we minimize the JS and tries to minimize the distance between two distributions, actually we try to minimize the distance between the entropy between the sum of two distribution with one of them, but what is the intuition behind it?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.