I still don't understand how we can approximate the gradient of an expected value... Indeed it's impossible to sample points and then to average the gradients of them as we have only samples... (How to compute derivatives of samples...?)
The log-derivative trick seems to resolve this issue, and i have read that it allows you to compute Monte Carlo estimate on expressions that were untractable before...
If we recall the formula :
I agree that it's impossible to track the first expression with a Monte Carlo as the gradient of p(theta) is not a distribution. But why is it now possible to track the expectation of p(theta) * grad(log(p(theta))) ? What is the crucial changement ?
Thanks a lot for your potential answers !