# Why do we use the log-derivative trick before Monte Carlo?

I still don't understand how we can approximate the gradient of an expected value... Indeed it's impossible to sample points and then to average the gradients of them as we have only samples... (How to compute derivatives of samples...?)

The log-derivative trick seems to resolve this issue, and i have read that it allows you to compute Monte Carlo estimate on expressions that were untractable before...

If we recall the formula : I agree that it's impossible to track the first expression with a Monte Carlo as the gradient of p(theta) is not a distribution. But why is it now possible to track the expectation of p(theta) * grad(log(p(theta))) ? What is the crucial changement ?

• Did you mean $p(x, \theta)$? Then the gradient could be with respect to $\theta$ while the expectation is taken over $x$. This is the formulation for SGD-based variational Bayes, for example. Nov 14, 2019 at 2:09
• I do not know anything about MC but in other contexts like reinforcement learning they do the same trick. I think the crucial point is that one can write the RHS integral over $\theta$ as an expectation (but with respect to another random variable!) because now there is a term $p(\theta)$ which is not there in the left integral. However, if something is an expectation then we can approximate by using the law of large numbers and so forth. Nov 15, 2019 at 9:20
• There are some errors in the formula. The distribution should be $p(x \mid \theta)$ or $p(x; \theta)$. Also, integration is over $x$; there should be no $d\theta$ term. Nov 15, 2019 at 17:33