We are working on an optimization problem. The objective function involves distance between data points. We tried a wide variety of distance measures and found the entropy-based measures, especially the Jensen-Shannon divergence, perform really well. We did some analyses and found it is due to the nature of the data.

To explain our work, we will most likely encounter question about our use of Jensen-Shannon since it is typically employed to compute the distance between probability distributions.

Is there any reference that can show the entropy-based measures are also used for nonprobabilistic objects, e.g. simple high-dimensional data points?

  • $\begingroup$ Hi, I have the same problem, did you find any answer to your question? I would appreciate if you help me too. Thanks $\endgroup$
    – user137927
    Commented Aug 23, 2020 at 10:21
  • $\begingroup$ @user137927 We did not find any reference.. In the end, we just said the entropy measure has the best performance among all and explained the reason qualitatively. Met no resistance. If you had any luck, please let me know!:) $\endgroup$ Commented Aug 24, 2020 at 3:28
  • $\begingroup$ Sure, Many thanks for your reply :) $\endgroup$
    – user137927
    Commented Aug 24, 2020 at 12:44
  • $\begingroup$ Perhaps a bit late, but you may benefit from having a look at out paper: mdpi.com/2673-9984/5/1/22 $\endgroup$ Commented Apr 10 at 18:49


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.