1
$\begingroup$

I am aware there are various ways to calculate the Hellinger Distance (H) depending on the context and data. One of these ways, as I understand, is via the Bhattacharyya coefficient (BC). For discrete distributions, $H=\sqrt{1-BC} $ where $BC=\sum_{i=1}^n \sqrt{p_i q_i} $. Hence we have:

$$H=\sqrt{1-\sum_{i=1}^n \sqrt{p_i q_i}} $$

However, I have found some expressions of the Hellinger Distance equation that includes a factor of 2 (see page 302, here) in the form of:

$$H=2\sqrt{1-\sum_{i=1}^n \sqrt{p_i q_i}} $$ This is equivalent to $H=2\sqrt{1-BC} $ found in a Cross Validated question here.

So which version of the Hellinger Equation is correct? Or am I missing something? A factor of two in a distance measure is hardly a trivial difference.

Improve this question
$\endgroup$
10
  • 1
    $\begingroup$ In fact for most things that are done with distances it is a trivial difference, as distances are usually compared with each other (i.e., Hellinger distances between different pairs of objects) or used in techniques such as cluster analysis or multidimensional scaling which are invariant to multiplying the distance by a constant. In fact I'm struggling to come up with any application of Hellinger distances where it would make a difference whether there's a factor 2 or not. So these different formulae may just be around with the implicit understanding that they're equivalent. $\endgroup$
    – Christian Hennig
    Jan 28, 2022 at 10:33
  • $\begingroup$ @Christian Hennig. Yes, I suspected this was likely to be the case. However, it's all nice and good to treat this difference as some arbitrary scale factor. Yet, surely an equation is an equation, invariant or not. We know the equations for, let's say, entropy, the normal distribution and that 1+1=2. The Hellinger equation shouldn't get so 'free pass' to have an implicit rather than objective understanding, comparative or not. In fact, the comparative nature suggests standardization would be a desirable property. Surely, Stats/Math literature could be more, well, definitive here... $\endgroup$
    – Mari153
    Jan 28, 2022 at 11:35
  • $\begingroup$ That's fair enough, however one can also see it like this: Generally, mathematical notation should not be taken as universally defined, but rather be explicitly defined when used. Mathematics is not about fixed meanings of descriptors, but about what is implied by the definitions, whatever they are. I'm not sure what the story is in case of Hellinger, but it may for example be that one person came up with one definition, and someone else discovered that there is a nice theoretical motivation that adds in a certain scale factor and is therefore in some sense "better". (To be continued) $\endgroup$
    – Christian Hennig
    Jan 28, 2022 at 11:59
  • $\begingroup$ Also, this other person may have realised that for all conceivable applications it may not matter whether the scale factor is there or not. So now, which definition is "correct"? I'd say there is no answer (and whose job would it be to decide?). People who use it better write down explicitly which one they mean, which can be one or the other, then the reader knows and all is fine. (Of course one can argue that some things should be defined everywhere in the same way as one can hardly explain everything from scratch, so it's hard to draw the line, but still...) $\endgroup$
    – Christian Hennig
    Jan 28, 2022 at 12:01
  • $\begingroup$ One other example is that of information criteria AIC and BIC there are "larger is better" and "smaller is better"-versions (use of factor -1 or not) around. In some literature they won't even tell you explicitly which one they use, and I do agree that this is annoying, and it'd be nicer to have a unique one used by everyone, but then there's no instance to decide this, so we have to live with what's around (I teach my students to explicitly define all notation whenever possible). $\endgroup$
    – Christian Hennig
    Jan 28, 2022 at 12:06

1 Answer 1

Reset to default
1
+50
$\begingroup$

This doesn't really answer the question but maybe helpful anyway.

All applications of the Hellinger distance I can think of are invariant to whether there's a factor 2 in the definition or not, potentially adjusting, e.g., threshold values by the same factor. Obviously whatever version is used needs to be used consistently, so it is advisable that the used formula is always explicitly given when using the Hellinger distance.

For this reason, most mathematicians would consider the two versions equivalent. There is no consensus about which one is right, and there is no authority that would enforce such a consensus. Most mathematicians would think that no such consensus is needed, as the two are "the same" in all relevant aspects anyway.

Historically, one possibility for such a situation to emerge (I don't know about the Hellinger distance in particular) is that somebody defines a concept originally, and somebody else discovers that the same concept (but multiplied with a constant factor not present in the original definition) emerges nicely out of some theoretical considerations that help a lot motivating the concept; after which for both versions there is a reason to be seen as legitimate.

Generally, a mathematical way of looking at such things is that names and notation should not be taken as having a generally agreed meaning but rather they should be explicitly defined when used and then they are what they are defined to be, in the specific place.

It has to be admitted though that there are limitations to this attitude. Work of a certain complexity cannot define everything from scratch for pragmatic reasons, and non-mathematicians are understandably often baffled by the same name apparently not referring to the (exactly) same thing. So certain conventions are required and some exist (too many from the point of view of some pure mathematicians; not enough from the point of view of many other people).

As another example, personally I am annoyed to see that the BIC and AIC as used for model selection are used in some literature in the positive and in other literature in the negative form, so in one case "larger is better", in another one "smaller is better" - for sure the authors need to tell the readers explicitly which version is used, but in many places this is not done, and the reader has to guess from looking at reported results which one it is.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks Christian. I'll give you the bounty. Your answer is close enough for me and has pointed me in a good direction. I have uncovered some research that makes it clear that for the Hellinger Distance "different definitions are used frequently in the literature". For my two cents worth, based on what I have read and the associated proofs, I lean towards the equation without the 2-factor. $\endgroup$
    – Mari153
    Feb 2, 2022 at 11:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.