Variational Bayes combined with Monte Carlo I'm reading up on variational Bayes, and as I understand it, it comes down to the idea that you approximate $p(z\mid x)$ (where $z$ are the latent variables of your model and $x$ the observed data) with a function $q(z)$, making the assumption that $q$ factorizes as $q_i(z_i)$ where $z_i$ is a subset of the latent variables. It can then be shown that the optimal factor $q_i(z_i)$ is:
$$
q^*_i(z_i) = \langle \ln p(x, z)\rangle_{z/i} + \text{const.}
$$
Where the angle brackets denote the expectation over all latent variables except $z_i$ with respect to the distribution $q(z)$.
Now, this expression is usually evaluated analytically, to give an exact answer to an approximate target value. However, it occurred to me that, since this is an expectation, an obvious approach is to approximate this expectation by sampling. This would give you an approximate answer to an approximate target function, but it makes for a very simple algorithm, perhaps for cases where the analytical approach is not feasible.
My question is, is this a known approach? Does it have a name? Are there reasons why it might not work so well, or might not yield such a simple algorithm?
 A: I'll confess this isn't a domain I know very well, so take this with a grain of salt.
First of all, note that what you are proposing doesn't yield such a simple algorithm: in order to compute the new $q^\star_i$, we don't need to compute a single expected value (like a mean or variance), but the expected value of a whole function. This is computationally hard and will require you to approximate the true $q^\star$ by some $\tilde q$ (for example, we might find a histogram approximation)
But, if you are going to be restricting the $q_i$ to a small parametric family, a better idea might be to use stochastic gradient descent to find the best parameter values (see: Variational bayesian inference with stochastic search, 2012, Paisley, Blei, Jordan). The gradient they compute is very similar to what you wrote: they sample from all the approximations they are currently not optimizing.
So what you propose isn't that simple, but it's quite close to an actual method that has been proposed very recently
