With the definition of the KL divergence between a model probability density function (pdf) and the data pdf

$$D[p|q] = \bigg< log \frac{p(x)}{q(x)} \bigg>_p = \int_{-\infty}^{\infty} p(x) log \frac{p(x)}{q(x)}dx$$

taken from this paper*, I'm trying to show that the gradient descent on an arbitrary parameter $x$ is given by

$$\partial_x D[p|q] = \bigg < \bigg (1+log\frac{p(x)}{q(x)} \bigg) \partial_x log [p(x)] - \partial_x log[q(x)] \bigg >_p.$$

The derivative on $x$ gives me

$$\partial_x D[p|q] = \int_{-\infty}^{\infty} \bigg (\partial_xp(x)log\frac{p(x)}{q(x)}+ p(x)\partial_x log\frac{p(x)}{q(x)}\bigg)dx $$

but I'm unsure on how to proceed from here as I don't know how to approach the $\partial_xp(x)$ term in the equation above.

Is there a different way to arrive at the result or am I making a mistake here?

* Towards a cross-level theory of neural learning - AJ Bell - AIP Conference Proceedings, 2007

  • 1
    $\begingroup$ Can you add a full reference/citation to the linked paper? That way, if in future it gets taken offline or the link location changes (this often happens with an academic's university page if they move institution) we don't have a "link rot" problem, and people can still identify and read the paper. $\endgroup$ – Silverfish Oct 25 '16 at 16:41

\begin{align*} \partial_x D[p|q] &= \int_{-\infty}^{\infty} \bigg (\partial_xp(x)log\frac{p(x)}{q(x)}+ p(x)\partial_x log\frac{p(x)}{q(x)}\bigg)dx \\ &= \int_{-\infty}^{\infty}\partial_xp(x)log\frac{p(x)}{q(x)}+p(x)\frac{q(x)}{p(x)}\frac{\partial_xp(x)q(x)-p(x)\partial_xq(x) }{q(x)^2} dx \\ &= \int_{-\infty}^{\infty}\partial_xp(x)log\frac{p(x)}{q(x)}+q(x)\frac{\partial_xp(x)q(x)-p(x)\partial_xq(x) }{q(x)^2} dx \\ &= \int_{-\infty}^{\infty}\partial_xp(x)log\frac{p(x)}{q(x)}+\partial_xp(x)-\frac{p(x)}{q(x)}\partial_xq(x) dx\\ &= \int_{-\infty}^{\infty} \left[ \bigg (1+log\frac{p(x)}{q(x)} \bigg) \frac{\partial_x p(x)}{p(x)} - \frac{\partial_xq(x) }{q(x)}\right] p(x) dx\\ &= \bigg < \bigg (1+log\frac{p(x)}{q(x)} \bigg) \partial_x log [p(x)] - \partial_x log[q(x)] \bigg >_p \end{align*}


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.