# Minimising KL divergence between two distributions

Say, we want to approximate a distribution $$p(x)$$ with $$q(x|\theta)$$. We do not know the distribution $$p(x)$$ but we can draw samples from $$p(x)$$. The KL divergence between the two distributions is \begin{aligned} KL(p\Vert q) &= -\int \ln\frac{q(x)}{p(x)}p(x)dx\\ \end{aligned} We can approximate the above quantity by drawing samples from $$p(x)$$. Replacing the above quantity with expectation, we get $$KL(p\Vert q)=\frac{1}{N}\sum_{i=1}^N\{-\ln \,q(x_i|\theta) + \ln\,p(x_i)\}$$ Minimising this quantity with respect to $$\theta$$ is the same as maximising the likelihood of $$q(x_i|\theta)$$.

However, Christopher Bishop writes it as just $$\sum_{i=1}^N\{-\ln \,q(x_i|\theta) + \ln\,p(x_i)\}$$. Shouldn't there be a $$1/N$$ in front ?

$$\frac{1}{N}$$ is just a constant that multiplies your objective function. As such, it does not affect the location of the interest points (minimums, maximums) with respect to your parameter $$\theta$$.