Posterior estimation using VAE

Question

Using normalizing flows, we can model model's posteriors $p(\theta|D)$, by feeding Gaussian noise $z$ to the NF (parametrized with $\phi$), using the output of the NF $\theta$ as model parameters, and then computing the loss. At that point, we can compute $\nabla \log p(x) = \frac{dL}{d\theta}\frac{d\theta}{d\phi}$, to which we add the log-determinant to account for the change of variable.

At that point, the NF will produce $\theta\sim p(\theta|D)$, so we successfully have approximated the posterior using a NF

However, I was wondering if this is a special case of NF, or if any generative model is able to do so. However, trying to fit this framework into the VAE world, I'm not so sure how to approach it.

The problem I'm facing with this reasoning, is that VAEs require to have samples from the intended distribution, and then just "make such samples likely under the model", where instead the NF is (in very poor words) a change of variable

---

Maybe something like this, with importance sampling: $$ \max_\theta \mathbb{E}_{p(x)}[\log p_\theta(x)] = \max_\theta \mathbb{E}_{q(x)}[\frac{p(x)}{q(x)}\log p_\theta(x)] $$ Instead of $\log p(x)$ we would put ELBO

so like if we have a set of weights $\theta$, we feed it through the VAE, calculate the ELBO, and weight each sample loss by the posterior likelihood (loss produced by that set of weights)

Answer 1

NF has strict requirement for the invertible transformation flow where not every complex target distribution can satisfy, and thus it's mainly used for density estimation when above requirement can be met in practice. In this sense NF as the main generative model is unique and most other models are not special case of it. Yet this doesn't mean it cannot be leveraged as a component in a VAE framework, in fact, the change of variable in NF as you described allows you to model distributions in a much more flexible and accurate way than the simple dimension-reduced Gaussian posterior in VAEs which many believe causes blurry images. Also this additional invertible parameterized flow to compute the approximated posterior in encoder is very akin to the reparameterization trick.

Your intended importance sampling to weight each generated sample's reconstruction loss in the decoder, say via the ratio of a prior Gaussian and the approximated NF posterior for the same sampled latent variable $z_i$, is required in the sense that you need to weight the ELBO based on how well each generated sample from a very complicated potentially multi-modal approximated NF posterior matches the true posterior, though VAE doesn't typically do this due to the conjunction of the simplicity of its variational posterior's assumed Gaussianity with respect to which the ELBO takes expectation and the KL-term.

In fact there're some papers studying flow based VAEs you may further refer, such as Su et al's (2018) f-VAEs: Improve VAEs with Conditional Flows.

In this paper, we integrate VAEs and flow-based generative models successfully and get f-VAEs. Compared with VAEs, f-VAEs generate more vivid images, solved the blurred-image problem of VAEs. Compared with flow-based models such as Glow, f-VAE is more lightweight and converges faster, achieving the same performance under smaller-size architecture.