# Minimisation of Kullback-Leibler divergence on an arbitrary parameter

With the definition of the KL divergence between a model probability density function (pdf) and the data pdf

$$D[p|q] = \bigg< log \frac{p(x)}{q(x)} \bigg>_p = \int_{-\infty}^{\infty} p(x) log \frac{p(x)}{q(x)}dx$$

taken from this paper*, I'm trying to show that the gradient descent on an arbitrary parameter $x$ is given by

$$\partial_x D[p|q] = \bigg < \bigg (1+log\frac{p(x)}{q(x)} \bigg) \partial_x log [p(x)] - \partial_x log[q(x)] \bigg >_p.$$

The derivative on $x$ gives me

$$\partial_x D[p|q] = \int_{-\infty}^{\infty} \bigg (\partial_xp(x)log\frac{p(x)}{q(x)}+ p(x)\partial_x log\frac{p(x)}{q(x)}\bigg)dx$$

but I'm unsure on how to proceed from here as I don't know how to approach the $\partial_xp(x)$ term in the equation above.

Is there a different way to arrive at the result or am I making a mistake here?

* Towards a cross-level theory of neural learning - AJ Bell - AIP Conference Proceedings, 2007

• Can you add a full reference/citation to the linked paper? That way, if in future it gets taken offline or the link location changes (this often happens with an academic's university page if they move institution) we don't have a "link rot" problem, and people can still identify and read the paper. – Silverfish Oct 25 '16 at 16:41