4
$\begingroup$

The following is from Section 2.2 of the Auto-Encoding Variational Bayes paper,

enter image description here

It says the gradient of the lower bound w.r.t $\phi$ is a bit problematic because the Monte Carlo estimator exhibits very high variance.

Why? Is it because sampling from $q_\phi(z|x)$ is of high variance? What about the gradient w.r.t $\theta$? $$\nabla_\theta E_{q_\phi(z|x)}[\log p_\theta(x|z)]=E_{q_\phi(z|x)}[\nabla_\theta\log p_\theta(x|z)]$$ Why is this not described as problematic?

A note to myself
My original confusion was that since the expectation is essentially an integral, $$\int_z q_\phi(z)f_\theta(z)$$ why its derivative w.r.t $\phi$ is more difficult than the derivative w.r.t $\theta$?

Now I seem to understand that the gradient w.r.t $\theta$ can be approximated approximate it by sampling from distribution $q_\phi(z)$, whereas $f_\theta(z)$ is not a distribution so we cannot do the same for $\phi$.

As an alternative we may use the log derivative trick as mentioned in the paper, but it's usually of high variance (as mentioned in the accepted answer).

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

If I understand correctly, sampling from $q_\phi(z|x)$ is not difficult: it's just a draw from a Gaussian distribution whose mean and variance are specified by $\phi$.

The problem is specifically with marginalizing out $\phi$ using samples from $q_\phi$. Radford Neal has a great blog post on why this can be a bad idea. Apart from the large (possibly infinite) variance of the estimate (assuming finite Monte Carlo samples), the estimate is also very sensitive to tiny changes in $q$'s distribution.

Improve this answer
$\endgroup$
2
  • $\begingroup$ thank you much, I'm not very familiar with this though. do you mean marginalizing out $z$ using samples from $q_\phi$? and the problem is $q_\phi$ is too narrow (like the posterior in Radford's blog)? so it takes lots of samples to cover the regions where $q_\phi(z|x)$ is small but $q_\phi(z|x)f(z)\nabla\log q_\phi$ can be large? $\endgroup$
    – dontloo
    Commented Oct 26, 2016 at 4:03
  • 1
    $\begingroup$ Sorry, yes. Marginalizing out $z$ using samples from $q_\phi$. The gradient of $\log(q_\phi)$ with respect to $\phi$ is $1/q_\phi$, so we're in the same "harmonic mean" territory discussed in the blog post. Values in the tails that are extremely unlikely to show up in a finite sample nevertheless make huge contributions to the final result. $\endgroup$
    – David J. Harris
    Commented Oct 31, 2016 at 18:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.