6
$\begingroup$

I've been searching but I couldn't find on the internet if there's any significance for the integral of the pdf squared:

$\int_\mathbb{R} f^2(x)$

That's because, as an alternative of Shannon entropy, you could measure the entropy (or in this case the "order") of a pdf as its distance to the uniform distibution pdf. As it is a "measure" of disimilarity, distance in either way is positive, so, for a discrete probability distribution, its order measure would be defined as:

$\sum_{i}^\mathbb{R} ( p(i) - \frac{1}{\mathbb{R}} )^2 $

A problem with this would be that the order of a discrete distribution would be affected by its range, but if the same formula is considered in a continuous and infinite range, it becomes:

$\int_\mathbb{R} ( f(x) - \frac{1}{\mathbb{R}})^2$

As it is evaluated in an infinite range it becomes:

$\int_\mathbb{R} f^2(x)$

I wanted to know if this would be a valid measure of the entropy (0 for uniform, infinite for something like the dirac delta function) or if there's a flaw I'm not considering for using it in that context.

$\endgroup$
2
  • 2
    $\begingroup$ You might want to have a look at the Gini entropy that is a special case of the Tsallis entropy $\endgroup$
    – Simone
    Commented Nov 19, 2017 at 3:27
  • 1
    $\begingroup$ Thanks! From what I've read so far, the continuous formula for Gini entropy fits the same criteria. $\endgroup$
    – santiago
    Commented Nov 19, 2017 at 20:18

1 Answer 1

2
$\begingroup$

This is a continuous version of what in the discrete case is called the Simpson index $\sum_i p_i^2$, see Derivation of the Simpson index This is a kind of diversity index, and such an interpretation should be possible in the continuous case also.

Literally, it is the expected value of the probability density function at the observed value of the random variable.

$\endgroup$
1
  • 1
    $\begingroup$ These are good comments, but I am concerned about how this integral is not independent of scale: that would seriously complicate its interpretation as any kind of diversity index. How does one cope with that? $\endgroup$
    – whuber
    Commented Aug 10, 2023 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.