Question about the implementation of variational inference I'm studying variational inference and trying to implement the algorithm. But I came across a question when computing the ELBO. 
Suppose for the latent variable $z$, I have a variational distribution $q_w(z)$, $w$ is the parameter I want to infer. The joint distribution of data and $z$ is given as $P(Data, z)$.
So the ELBO is 
$ELBO = \int_z q_w(z)\log P(Data, z) d_z - \int_z q_w(z)\log q_w(z)d_z$.
When I'm implementing the algorithm for the first part $\int_z q_w(z)\log P(Data, z) d_z$, I know that I'm going to draw a sample $r$ from $q_w(z)$ first. But do I need to go back to compute the likelihood $q_w(r)$, then use the likelihood times $\log P(Data, r)$? Or I can simply compute $\log P(Data, r)$ and ignore the $q_w(r)$ term?
 A: In the Monte Carlo estimation principle we deal with intractable integrals by means of sampling. In particular, the integral $$ L := \int q_w(z) \log P(\text{Data}, z) dz $$is too hard to compute. Luckily, it has the form of an average of some function (namely, $\log P(\text{Data}, \cdot)$) with weights $q_w(z)$. We thus can form a stochastic estimate of this quantity: $$ \hat{L}_M := \frac{1}{M} \sum_{m=1}^M \log P(\text{Data}, z_m), \quad \text{where} \;\; z_1, \dots, z_M \stackrel{i.i.d.}{\sim} q_w(z) $$
Now, $\hat{L}_M$ is a random variable, whereas $L$ originally was a number. However, we can show that $\hat{L}_M$ revolves around $L$, namely, expected value of $\hat{L}_M$ is equal to $L$ (for any $M$), and the variance (a measure of how much we tend to miss $L$) decreases as we take more samples $M$:
$$
\mathbb{E}\left[\hat{L}_M\right] = L,
\quad\quad
\mathbb{V}\left[\hat{L}_M\right] = \frac{1}{M} \mathbb{V}\left[\log P(\text{Data}, z_1)\right]
$$
Obviously, the bigger the $M$ is – the more precise estimate of $L$ you get. But this come at a price of your algorithm become increasingly computationally expensive (you'd need to perform $M$ independent calculations!). In stochastic optimization people often use $M = 1$ and deal with the incurred variance with (1) decreasing learning rate, (2) performing more optimization steps.
Notice the absence of any weighting factors $q_w(z)$ in $\hat{L}_M$. This is because these weights have been incorporated by the sampling procedure. The bigger the weight $q_w(r)$ for some $r$ – the more often this particular $r$ (or, more precisely, values from its vicinity) will be realized when sampled.
Finally, Monte Carlo method is, I'd say, a fundamental tool in many scientific and engineering disciplines. I encourage you to learn more (you can start with the wikipedia).
