2
$\begingroup$

In the context of variational inference, we move from estimating the gradient of:

$\log E_{q \sim Q(z|x)} [ \frac{p(z,x)}{q(z|x)} ]$

to: $E_{q \sim Q(z|x)} [ \log \frac{p(z,x)}{q(z|x)} ]$, which is the ELBO.

I'd like to explicitly compare the variances between a Monte Carlo estimator for the gradients of these two quantities, but am struggling with the algebra.

By Monte Carlo estimator, I mean we replace the expectations in each expression with the sample mean of the random variables inside each expectation.

Can you help?

Reference: https://arxiv.org/pdf/2208.11970, page 3, eq7-8

Basically I don't understand why we don't maximize equation 7 in the above reference instead of 8. Perhaps it is because the variance is lower, but how to show that rigorously?

$\endgroup$
2
  • $\begingroup$ The variance does not have to be smaller! Actually, since$$E_{Z \sim Q(z|x)} [ \frac{p(Z,x)}{q(Z|x)}] =p(x)$$ the first term has a variance of zero! The use of the expected log is due to the separation properties of the Kullback divergence that make computing it more easily while being a natural discrepancy between distributions. $\endgroup$
    – Xi'an
    Commented May 27 at 12:20
  • 1
    $\begingroup$ "The use of the expected log is due to the separation properties of the Kullback divergence that make computing it more easily while being a natural discrepancy between distributions. " Can you elaborate on your answer? $\endgroup$
    – user3180
    Commented May 28 at 21:33

1 Answer 1

2
$\begingroup$

First, let me copy the definitions of eq 7 and eq 8 here for completeness:

$$ \begin{align*} \log p(x) & = \log \mathop{\mathbb{E}}_{q(z|x)} \frac{p(x, z)}{q(z|x)} \tag{7} \\ & \ge \mathop{\mathbb{E}}_{q(z|x)} \log \frac{p(x, z)}{q(z|x)} \tag{8} \end{align*} $$

We cannot directly use the eq. (7) because we cannot compute the expectation precisely.

Sure, we could estimate it, but a constant 0 would also be an estimator albeit an extremely poor one. To be of practical use we want our estimators to have certain guarantees, in particular we care about bias and variance.

So given a sample $z_1, \dots, z_N$ from $q(z|x)$ let's investigate the following estimator of (7): $$ G_\text{log-avg} := \log \frac{1}{N} \sum_{n=1}^N \frac{p(x, z_n)}{q(z_n|x)} $$

First, $G_\text{log-avg}$ is a biased estimator, that in general it's mean is not the quantity we seek to estimate:

$$ \begin{align*} \mathop{\mathbb{E}}_{q(z_{1:N}|x)} G_\text{log-avg} &= \mathop{\mathbb{E}}_{q(z_{1:N}|x)} \log \frac{1}{N} \sum_{n=1}^N \frac{p(x, z_n)}{q(z_n|x)} \\ &\le \log \left [\mathop{\mathbb{E}}_{q(z_{1:N}|x)} \frac{1}{N} \sum_{n=1}^N \frac{p(x, z_n)}{q(z_n|x)} \right] \\ &= \log \left [\mathop{\mathbb{E}}_{q(z_1|x)} \frac{p(x, z_1)}{q(z_1|x)} \right] \end{align*} $$ Where the inequality is due to log's concavity and the last line is due to $z_n$ being distributed identically. The equality is achieved if and only if $\frac{1}{N} \sum_{n=1}^N \frac{p(x, z_n)}{q(z_n|x)}$ is constant w.r.t. all $z_n$, which requires $q(z_n|x) = p(z_n|x)$ everywhere, which usually is not feasible.

So it turns out $G_\text{log-avg}$ is a biased estimate of $\log \mathbb{E}$, whereas the $\mathbb{E} \log$ in eq. 8 can be readily estimated, by the very definition of Monte Carlo estimation:

$$ G_\text{avg-log} := \frac{1}{N} \sum_{n=1}^N \log \frac{p(x, z_n)}{q(z_n|x)} $$

$$ \mathop{\mathbb{E}}_{q(z_{1:N}|x)} G_\text{avg-log} = \mathop{\mathbb{E}}_{q(z_{1:N}|x)} \frac{1}{N} \sum_{n=1}^N \log \frac{p(x, z_n)}{q(z_n|x)} = \mathop{\mathbb{E}}_{q(z_1|x)} \log \frac{p(x, z_1)}{q(z_1|x)} $$

So naturally eq. 8 is easier to get an unbiased estimate of, something we care about especially when it comes to stochastic optimization.

However, in a plot twist it turns out $G_\text{log-avg}$ is a better estimate for the problem we have at hand than $G_\text{avg-log}$ is. The core observation is that while $G_\text{avg-log}$ is an unbiased estimator of eq. 8 we don't really care about $\mathbb{E} \log \frac{p(x,z)}{q(z|x)}$, as it's $\log p(x)$ that we seek to maximize. That $\log p(x)$ does coincide with eq. 7, so in that regard both $G_\text{log-avg}$ and $G_\text{avg-log}$ fail at the task of being an unbiased estimator of $\log p(x)$ that we seek. But it turns out the $G_\text{log-avg}$ has smaller bias than $G_\text{avg-log}$ does:

$$ \begin{align*} \mathop{\mathbb{E}}_{q(z_{1:N}|x)} G_\text{avg-log} &= \mathop{\mathbb{E}}_{q(z_{1:N}|x)} \frac{1}{N} \sum_{n=1}^N \log \frac{p(x, z_n)}{q(z_n|x)} \\ &\le \mathop{\mathbb{E}}_{q(z_{1:N}|x)} \log \frac{1}{N} \sum_{n=1}^N \frac{p(x, z_n)}{q(z_n|x)} \\ &= \mathop{\mathbb{E}}_{q(z_{1:N}|x)} G_\text{log-avg} \\ &\le \log \left [\mathop{\mathbb{E}}_{q(z|x)} \frac{p(x, z)}{q(z|x)} \right] = \log p(x) \end{align*} $$

Where the first inequality comes from log's concavity once again and the second one has been established before.

So we have just shown that $G_\text{log-avg}$ is closer to $\log p(x)$ than $G_\text{avg-log}$. It can also be shown to have lower variance. The catch however is that it requires $N$ times more compute: $G_\text{avg-log}$ can be estimated using just one sample from $q(z|x)$ whereas $G_\text{log-avg}$ requires $N$ samples (and they coincide when $N=1$, so bigger $N$ is preferable). Finding a balance between quality and efficiency becomes a separate task in itself.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.