# Questions tagged [variational-bayes]

Variational Bayesian methods approximate intractable integrals found in Bayesian inference and machine learning. Primarily, these methods serve one of two purposes: Approximating the posterior distribution, or bounding the marginal likelihood of observed data.

394 questions
Filter by
Sorted by
Tagged with
19 views

### Why do we say that we're "predicting" the mean/noise in diffusion models?

In DDPM, ${\tilde\mu}_t$ is the mean of the conditional distribution $q(x_{t-1}|x_t,x_0)$ while the neural network $\mu_\theta$ is modeling a different conditional distribution $p_\theta(x_{t-1}|x_t)$....
44 views

### Why is the forward process referred to as the "ground truth" in diffusion models?

I've seen in many tutorials on diffusion models refer to the distribution of the latent variables induced by the forward process as "ground truth". I wonder why. What we can actually see is ...
129 views

### Theoretical justification for minimizing $KL(q_\phi|p)$ rather than $KL(p|q_\phi)$?

Suppose we have a true but unknown distribution $p$ over some discrete set (i.e. assume no structure or domain knowledge), and a parameterized family of distributions $q_\phi$. In general it makes ...
• 2,877
31 views

### Prove that variance of log Exp is greater than variance of Exp log

In the context of variational inference, we move from estimating the gradient of: $\log E_{q \sim Q(z|x)} [ \frac{p(z,x)}{q(z|x)} ]$ to: $E_{q \sim Q(z|x)} [ \log \frac{p(z,x)}{q(z|x)} ]$, which is ...
• 591
23 views

### How to do prediction (evaluate marginal likelihood) in generative latent variable classifier?

The dataset is $\{\boldsymbol x_t,y_t\}$ for $t=1,\dots,T$, where $y_t \in \{0,1\}$. Define a generative latent variable classifier whose plate diagram is shown above. For each data point, a local ...
• 81
95 views
+50

### Why don’t diffusion models suffer posterior collapse?

In VAEs, posterior collapse occurs when the approximated posterior $q_\theta(z|x)$ becomes the standard Gaussian prior $p(z)$ after training (Lucas et al. 2019). The forward process of diffusion ...
1 vote
42 views

• 81
24 views

### Combining MCMC with Variatonal Inference

I have a Gibbs sampler that is mixing terribly slowly. I have a hunch that if I sample a parameter pair as a single block, it would improve convergence. I tried HMC within Gibbs, but it's also slow. I ...
52 views

88 views

### Which is the best way to implement variational inference?

To implement variational inference in a Bayesian model, one essentially has the choice between different approaches that differ in their degree of automation and flexibility: manually deriving update ...
• 21
1 vote
56 views

### Initialization of parameters of Bayesian Neural Network

What is the best practice to initialize parameters of the Bayesian Neural Network? I am stuck in a situation where I don’t initialize the parameters and every time I run the variational inference, I ...
• 225
237 views

### How can be decoder of VAE represent probability distribution p(x|z) eventhough we directly get image as output

How can be decoder of VAE represent probability distribution p(x|z) eventhough we directly get image as output. Also if my current understanding is right than we get same image for same value of z ...
963 views

### How to Resolve Variational Autoencoder (VAE) Model Collapse in Reconstruction Task Using Sensor Data?

I am currently experiencing a suspected model collapse in a Variational Autoencoder (VAE) model I am working with. Below are details on the project setup and the issue at hand: Project Goal: Exploring ...
1 vote
30 views

### VAE: Noisy decoder output

I'm trying to implement a simple VAE by following several tutorials like this and this. This is the code that I came up with: ...
• 11
1 vote
48 views

### How does Variational Autoencoder approximate the joint probability distribution?

I know that in Variational Inference the idea is to approximate the posterior P(z|x, y) and I know that Variational AutoEncoders (VAEs) use the idea of variational inference through neural network ...
269 views

### Conceptual questions about the proxy distribution in variational inference

I am trying to implement a variational extension of some kind of Bayesian network estimation method. The main goal is to improve speed, since the current method is pretty slow due to MCMC. My question ...
1 vote
50 views

### Prior term in SGHMC implementation

I am working with SGHMC (Stochastic Gradient Hamiltonian Monte Carlo) models. I found an implimentation of the algorithm in pytorch here. The part of the code that represents momentum variable update (...
147 views

### What is a Student-t VAE ? and how is it different from Gaussian VAE?

I am currently reading https://www.ijcai.org/proceedings/2018/0374.pdf ,this is a research paper based on Student-t Variational Autoencoder for Robust Density Estimation , In this research paper, they ...
• 31
1 vote
40 views

• 1,366
505 views

### Is VAE based on mean-field assumption?

According to the slides (section 6), mean-field variational inference (aka. MFVI) assumes the latent variables ($z=\{z_1,..,z_m\}$ are independent from each other, and on top of this assumption, we ...
• 3,633
463 views

### What is the closed-form of the KL-Divergence between two relaxed Bernoulli distributions?

I've seen in multiple papers that use a relaxation of the Bernoulli distribution as defined in Maddison et. al (here it is referred to as Binary Concrete) and they say that a closed form solution for ...
52 views

### How does the decoder predict directly the image in VAEs?

I am reading the VAEs paper Auto-Encoding Variational Bayes. In their loss function: they define reconstruction loss (second RHS term) as the expected value of the log p(x|z) wrt to the posterior of ...
• 61
190 views

### KL divergence between GPs undefined?

In Theorem 1 in Sun et al, Functional Variational Bayesian Neural Networks, 2019, the authors state the the KL divergence between stochastic processes in the supremum over KL divergence between ...
• 33
179 views

### Mean field approximation convergence

The last sentence of Christopher M. Bishop, Pattern Recognition and Machine Learning Section 10.1.1 Factorized distributions on p.466, states, referring to Equation $(10.9)$, that "Convergence is ...
• 1,015
1 vote
I am reading about the reparameterization trick from here. It states $\boldsymbol{\epsilon}\sim p(\boldsymbol{\epsilon})$, $\textbf{z}=g_\theta(\boldsymbol{\epsilon},\textbf{x})$, and \mathbb{E}_{p_\...