# Renyi divergence identity

I'm reading the paper, T. van Erven and P. Harremoës, Rényi Divergence and Kullback-Leibler Divergence, arXiv 1206.2459 on the Renyi divergence, and I'm trying to make sense of "Example 1". I think that I'm just getting messed up on the terminology, so I'm not able to derive that the identity is true.

Am I missing some assuptions/definitions/notational conventions so that this doesn't make sense?

Is there a different way of describing/deriving their "Example 1"?

Example 1: Let $Q$ be a probability distribution and $A$ a set with positive probability.
Let $P$ be the conditional distribution of $Q$ given $A$.
Then $$D_\alpha( P \Vert Q ) = - \ln Q(A).$$

Some features of their notation (from earlier in the paper): $$P=(p_1, p_2, \dots, p_n) \\ Q=(q_1, q_2, \dots, q_n)$$ refer to (discrete) probability distributions.

The Renyi divergence itself is expressed as: $$D_\alpha ( P \Vert Q ) = \frac{1}{1-\alpha} \ln \sum_{i=1}^{n} p_i^{\alpha} q_i^{1-\alpha}$$

I don't understand the notion of "the conditional distribution of $Q$ given $A$", to me, this reads as "the probability distribution of a probability distribution [function]".

I'm also unclear on the interpretation the result: the result should be the number, but what's written (looks like it) is the log of a set (note the definition of $A$).

## 1 Answer

By "the conditional distribution of $Q$ given $A$" they mean: $$p_i = \frac{q_i}{\sum_{i\in A} q_i}\text{ if } i \in A$$ and zero otherwise. The denominator is just normalization, so that $p_i$ sum up to $1$.

With that defined $P$ and $Q$ the divergence yields $$\frac{1}{1-\alpha}\ln\sum_{i\in A} \left( \frac{q_i^\alpha}{(\sum_{j\in A} q_j)^{\alpha}}q_i^{1-\alpha} \right)$$ and separating the logarithm with respect to two sums we get $$\ln \sum_{i\in A}q_i.$$ So, $Q(A)$ is probability of $Q$ being in $A$ (or, equivalently, probability summed over set $A$).