With the definition of the KL divergence between a model probability density function (pdf) and the data pdf

$$D[p|q] = \bigg< \log \frac{p(x)}{q(x)} \bigg>_p = \int_{-\infty}^{\infty} p(x) \log \frac{p(x)}{q(x)}dx$$

taken from this paper*, I'm trying to show that the gradient descent on an arbitrary parameter $x$ is given by

$$\partial_x D[p|q] = \bigg < \bigg (1+\log\frac{p(x)}{q(x)} \bigg) \partial_x \log [p(x)] - \partial_x \log[q(x)] \bigg >_p.$$

The derivative on $x$ gives me

$$\partial_x D[p|q] = \int_{-\infty}^{\infty} \bigg (\partial_xp(x)\log\frac{p(x)}{q(x)}+ p(x)\partial_x \log\frac{p(x)}{q(x)}\bigg)dx $$

but I'm unsure on how to proceed from here as I don't know how to approach the $\partial_xp(x)$ term in the equation above.

Is there a different way to arrive at the result or am I making a mistake here?

* Towards a cross-level theory of neural learning - AJ Bell - AIP Conference Proceedings, 2007

  • 1
    $\begingroup$ Can you add a full reference/citation to the linked paper? That way, if in future it gets taken offline or the link location changes (this often happens with an academic's university page if they move institution) we don't have a "link rot" problem, and people can still identify and read the paper. $\endgroup$
    – Silverfish
    Commented Oct 25, 2016 at 16:41

1 Answer 1


\begin{align*} \partial_x D[p|q] &= \int_{-\infty}^{\infty} \bigg (\partial_xp(x)\log\frac{p(x)}{q(x)}+ p(x)\partial_x \log\frac{p(x)}{q(x)}\bigg)dx \\ &= \int_{-\infty}^{\infty}\partial_xp(x)\log\frac{p(x)}{q(x)}+p(x)\frac{q(x)}{p(x)}\frac{\partial_xp(x)q(x)-p(x)\partial_xq(x) }{q(x)^2} dx \\ &= \int_{-\infty}^{\infty}\partial_xp(x)\log\frac{p(x)}{q(x)}+q(x)\frac{\partial_xp(x)q(x)-p(x)\partial_xq(x) }{q(x)^2} dx \\ &= \int_{-\infty}^{\infty}\partial_xp(x)\log\frac{p(x)}{q(x)}+\partial_xp(x)-\frac{p(x)}{q(x)}\partial_xq(x) dx\\ &= \int_{-\infty}^{\infty} \left[ \bigg (1+\log\frac{p(x)}{q(x)} \bigg) \frac{\partial_x p(x)}{p(x)} - \frac{\partial_xq(x) }{q(x)}\right] p(x) dx\\ &= \bigg < \bigg (1+\log\frac{p(x)}{q(x)} \bigg) \partial_x \log [p(x)] - \partial_x \log[q(x)] \bigg >_p \end{align*}


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.