# Conditional vs. nonconditional variational family

In variational bayes, distributions in the variational family $$\mathcal{Q}$$ are denoted $$q_\phi$$ and are used to approximate the posterior $$p_\theta(z|x)$$. However, I've seen both notations $$q_\phi(z|x)$$ and $$q_\phi(z)$$. What's the difference and how are they each used?

My interpretation was that $$q_\phi(z)$$ and $$q_\phi(z|x)$$ are a prior and posterior on $$\mathcal{Q}$$. Is this the right interpretation? It doesn't make sense that we should need a prior for $$q_\phi$$ since it is being optimized as an approximation to $$p_\theta(z|x)$$.

"My interpretation was that $$q_{\phi}(𝑧)$$ and $$q_{\phi}\left(z | x \right)$$ are a prior and posterior on $$\mathcal{Q}$$" is not correct at all. $$\mathcal{Q}$$ is the variational family defining the distribution space. $$q_{\phi}(𝑧)$$ and $$q_{\phi}\left(z | x \right)$$ live in $$\mathcal{Q}$$ and used to approximate the prior/posterior defined for a data distribution.

Both notations are used actually. $$q_{\phi}(𝑧)$$ actually stems from the ELBO derivation(You can check these lecture notes from CMU), i.e. the KL divergence definition.

For VAE's, we specifically denote $$q_{\phi}\left(z | x \right)$$ because the encoder tries to learn an amortized posterior distribution, which instead of optimizing a set of free parameters unlike in the lecture notes that I've shared sbove, we can propose parameterized function(like a neural network) that maps from observation space to the parameters of the approximate posterior distribution.

In fact, there are advanced VI techniques that allow implicit priors for VAE's like Semi-Implicit Variational Inference (SIVI).

TL;DR, try to stick with $$q_{\phi}\left(z | x \right)$$ for VAE posteriors and always make sure to understand which distribution you are trying to approximate.

Hope this helps.

• Thanks for your helpful answer! So why is amortized inference called that? Is it the fact that we aren't computing $q_\phi(z_i)$ for each individual datapoint but rather defining a global function and then letting each datapoint contribute? ($\leftarrow$ is this right?) – 900edges Mar 23 at 12:50
• Ah I did some reading online, will edit your answer for my own reference if that's okay with you – 900edges Mar 23 at 12:52
• @900edges There's nothing wrong with writing your own answer to the question. – Sycorax Mar 23 at 13:49
• @Sycorax I want to accept their answer, just wanted to fill out a detail! – 900edges Mar 23 at 13:52

Note: the point of Amortized VI is this:

Traditional VI defines the latent variable $$z_i$$ for each individual observation, and then optimizes them all jointly. This is computationally expensive to have to re-run the inference procedure for every datapoint. Instead, one can define a parameterized function, i.e. $$q_\phi(z|x)$$ sending $$x \rightarrow z$$. Now the parameters to update are the parameters of the function rather than the distributions of every $$z_i$$. For a new observation, we only have to pass it through the network rather than update everything.

• Good follow-up. Thank you @900edges – Emir Ceyani Mar 28 at 22:23