Why do some definitions of the Kullback-Leibler divergence include extra terms $-p_i + q_i$? For example, kl_div() (in the Python scipy.special module) seems to define the Kullback-Leibler divergence as $$ \sum_i p_i \ln\frac{p_i}{q_i} - p_i + q_i. $$

The documentation says:

The origin of this function is in convex programming; see [1] for details. This is why the function contains the extra terms over what might be expected from the Kullback-Leibler divergence.

I don't have the referenced book at hand. What is the justification or motivation for the additional $-p_i + q_i$ terms?

Anti-closing note: This is not a question about software, but about the concept behind it.

Why do some definitions of the Kullback-Leibler divergence include extra terms $-p_i + q_i$? For example, kl_div() (in the Python scipy.special module) defines the Kullback-Leibler divergence as $$ \sum_i p_i \ln\frac{p_i}{q_i} - p_i + q_i. $$

The documentation says:

The origin of this function is in convex programming; see [1] for details. This is why the function contains the extra terms over what might be expected from the Kullback-Leibler divergence.

I don't have the referenced book at hand. What is the justification or motivation for the additional $-p_i + q_i$ terms?

Anti-closing note: This is not a question about software, but about the concept behind it.

Why do some definitions of Kullback-Leibler divergence include extra terms $-p_i + q_i$? For example kl_div() (in the Python scipy.special module) seems to define the Kullback-Leibler divergence as $$ \sum_i p_i \ln\frac{p_i}{q_i} - p_i + q_i. $$

The documentation says:

The origin of this function is in convex programming; see [1] for details. This is why the function contains the extra terms over what might be expected from the Kullback-Leibler divergence.

I don't have the referenced book at hand. Can someone provide a short justification/motivation for the additional $-p_i + q_i$ terms?

Anti-closing note: This is not a question about software, but about the concept behind it.

Why do some definitions of the Kullback-Leibler divergence include extra terms $-p_i + q_i$? For example, kl_div() (in the Python scipy.special module) seems to define the Kullback-Leibler divergence as $$ \sum_i p_i \ln\frac{p_i}{q_i} - p_i + q_i. $$

The documentation says:

The origin of this function is in convex programming; see [1] for details. This is why the function contains the extra terms over what might be expected from the Kullback-Leibler divergence.

I don't have the referenced book at hand. What is the justification or motivation for the additional $-p_i + q_i$ terms?

Anti-closing note: This is not a question about software, but about the concept behind it.

Anti-closing preamble: This is not a question about software, but about the concept behind it.

Python's kl_div() (in the scipy.special module) seems to define the Kullback-Leibler divergence as $$ \sum_i p_i \ln\frac{p_i}{q_i} - p_i + q_i. $$

The documentation says:

The origin of this function is in convex programming; see [1] for details. This is why the function contains the extra terms over what might be expected from the Kullback-Leibler divergence.

I don't have the referenced book at hand. Can someone provide a short justification/motivation for the additional $-p_i + q_i$ terms?

Why do some definitions of Kullback-Leibler divergence include extra terms $-p_i + q_i$? For example kl_div() (in the Python scipy.special module) seems to define the Kullback-Leibler divergence as $$ \sum_i p_i \ln\frac{p_i}{q_i} - p_i + q_i. $$

The documentation says:

The origin of this function is in convex programming; see [1] for details. This is why the function contains the extra terms over what might be expected from the Kullback-Leibler divergence.

I don't have the referenced book at hand. Can someone provide a short justification/motivation for the additional $-p_i + q_i$ terms?

Anti-closing note: This is not a question about software, but about the concept behind it.