Skip to main content
edited body
Source Link
Nick Cox
  • 59.5k
  • 8
  • 136
  • 212

Tsallis and RenyiRényi entropy is the same thing, up to some rescaling. All of them are functions of $\sum_i p_i^\alpha$, with the special case of $\alpha\to1$ giving Shannon entropy.

Look at Tom Leinster's "Entropy, Diversity and Cardinality (Part 2)", especially at the table comparing these properties.

In short:

  • RenyiRényi entropies are in $[0, \log(N)]$,
  • Tsallis entropies (called there $\alpha$-diversities) are in $[0, (1-N^{1-\alpha})/(1-\alpha)]$,
  • $\alpha$-cardinalities are in $[1, N]$.

Also, one more way to go is to use:

  • 1/cardinality, in $[\tfrac{1}{N}, 1]$,
  • just $\sum_i p_i^\alpha$, in $[\tfrac{1}{N^{\alpha-1}}, 1]$.

The later two have the advantage that no matter what is the $N$, they always end up in $[0, 1]$.

Tsallis and Renyi entropy is the same thing, up to some rescaling. All of them are functions of $\sum_i p_i^\alpha$, with the special case of $\alpha\to1$ giving Shannon entropy.

Look at Tom Leinster's "Entropy, Diversity and Cardinality (Part 2)", especially at the table comparing these properties.

In short:

  • Renyi entropies are in $[0, \log(N)]$,
  • Tsallis entropies (called there $\alpha$-diversities) are in $[0, (1-N^{1-\alpha})/(1-\alpha)]$,
  • $\alpha$-cardinalities are in $[1, N]$.

Also, one more way to go is to use:

  • 1/cardinality, in $[\tfrac{1}{N}, 1]$,
  • just $\sum_i p_i^\alpha$, in $[\tfrac{1}{N^{\alpha-1}}, 1]$.

The later two have the advantage that no matter what is the $N$, they always end up in $[0, 1]$.

Tsallis and Rényi entropy is the same thing, up to some rescaling. All of them are functions of $\sum_i p_i^\alpha$, with the special case of $\alpha\to1$ giving Shannon entropy.

Look at Tom Leinster's "Entropy, Diversity and Cardinality (Part 2)", especially at the table comparing these properties.

In short:

  • Rényi entropies are in $[0, \log(N)]$,
  • Tsallis entropies (called there $\alpha$-diversities) are in $[0, (1-N^{1-\alpha})/(1-\alpha)]$,
  • $\alpha$-cardinalities are in $[1, N]$.

Also, one more way to go is to use:

  • 1/cardinality, in $[\tfrac{1}{N}, 1]$,
  • just $\sum_i p_i^\alpha$, in $[\tfrac{1}{N^{\alpha-1}}, 1]$.

The later two have the advantage that no matter what is the $N$, they always end up in $[0, 1]$.

Source Link
Piotr Migdal
  • 5.9k
  • 3
  • 34
  • 72

Tsallis and Renyi entropy is the same thing, up to some rescaling. All of them are functions of $\sum_i p_i^\alpha$, with the special case of $\alpha\to1$ giving Shannon entropy.

Look at Tom Leinster's "Entropy, Diversity and Cardinality (Part 2)", especially at the table comparing these properties.

In short:

  • Renyi entropies are in $[0, \log(N)]$,
  • Tsallis entropies (called there $\alpha$-diversities) are in $[0, (1-N^{1-\alpha})/(1-\alpha)]$,
  • $\alpha$-cardinalities are in $[1, N]$.

Also, one more way to go is to use:

  • 1/cardinality, in $[\tfrac{1}{N}, 1]$,
  • just $\sum_i p_i^\alpha$, in $[\tfrac{1}{N^{\alpha-1}}, 1]$.

The later two have the advantage that no matter what is the $N$, they always end up in $[0, 1]$.