# Questions tagged [variational-bayes]

Variational Bayesian methods approximate intractable integrals found in Bayesian inference and machine learning. Primarily, these methods serve one of two purposes: Approximating the posterior distribution, or bounding the marginal likelihood of observed data.

344 questions
Filter by
Sorted by
Tagged with
1 vote
47 views

### Poor reconstructions from using sigmoid in last layer of variational autoencoder

I have trained a variational autoencoder (VAE) using Pytorch Lightning to reproduce images. Without sigmoid, reproductions are good. However, some output image ...
• 23
32 views

### Posterior predictive distribution: Sampling vs calculating

I am having trouble understanding how to make predictions with the posterior predictive distribution. Posterior predictive is $p(y|x,D)=\int p(y|x,\theta)p(\theta|D) d\theta$ where $D$ is the training ...
• 371
1 vote
41 views

### Understanding the Evidence Lower Bound (ELBO)

I am reading this tutorial about Variational Inference, which includes the following depiction of ELBO as the lower bound on log-likelihood on the third page. In the tutorial, $x_i$ is the observed ...
• 1,427
17 views

### Modifying Variational Inference to be robust to outliers?

Normally, for variational inference, you have some evidence data $Z$, you have some true distribution $P(X|Z)$, and you have a simpler parameterized distribution $Q(X|\theta)$, and you're trying to ...
• 389
1 vote
23 views

• 793
51 views

### Regression in Baysian settings

Assume we have the posterior distribution of this linear regression model $y = w^Tx$, $P(w | D,\theta)$, where $D = \{(x_i,y_i)\}_{i \in \{1,\dots,n\}}, n$ is the number of data instances, $\theta$ ...
• 231
51 views

### Can a bayesian neural network with independent normal variational distributions over weights and biases produce a multi-modal posterior?

For a project I am involved in, I am doing surrogate modelling. This means that I simulate data $\mathcal{D}=\{X,Y\}$ that is used to train some probabilistic non-linear regression model. The model is ...
1 vote
24 views

### The Role Variational Free Energy or Expected Lower Bound (ELBO) Plays as a Loss Function

In reference to variational free energy or expected lower bound, I found this sentence, "As one can easily see, the cost function tries to balance the complexity of the data P(D | w) and the ...
80 views

### In bayesian approach, what is the difference between full posterior and MAP [duplicate]

Consider a classic machine learning problem, which we want to solve using NN. And suppose that we want to use bayesian learning for that. In the bayesian approach the posterior is described as follows:...
• 1,033
1 vote
87 views

### Clarification of Equation for Variational Inference in Pattern Recognition and Machine Learning

I am looking at the derivation of variational inference and specifically the approach taken by Bishop in his book on page 465 as illustrated in the Figure below. The key step is the statement below ...
• 51
45 views

### Is outer product of marginal distribution the "best" mean-field approximation for a joint distribution?

I am certain this has been asked somewhere else, if that's the case, link me and close the thread. I am studying variational inference and mean-field approximation. All the explanations I come across ...
• 103
76 views

### In VAE, why are we approximating p(z|x) using q(z) and not q(z|x) [duplicate]

I am watching this lecture on VAE: https://www.youtube.com/watch?v=uaaqyVS9-rM&t=1507s and at 26:00, it is stated that the goal is the minimize the KL div. between the distribution we are trying ...
• 213
1 vote
31 views

### Approximating the posterior and learning the distribution over the weights after training

I am familiar with the methods in variational inference in which after training we have access to the distribution over the network's weights. This is necessary for estimating epistemic uncertainty. ...
193 views

### How do we get to the MSE in the loss function for a variational autoencoder?

Context: https://arxiv.org/pdf/1312.6114.pdf So if I start with this equation:  \mathcal{L}\left(\boldsymbol{\theta}, \boldsymbol{\phi} ; \mathbf{x}^{(i)}\right) \simeq \frac{1}{2} \sum_{j=1}^{J}\...
• 93
91 views

### Convergence of CAVI(Coordinate Ascent Variational Inference)

I was reading several resources on variational inference, and most of them stated that the CAVI algorithm converges to local maximum, and Bishop's textbook stated that the convergence is guaranteed as ...
1 vote
58 views

### I am confused about the sum indices of these posterior distribution formulas

I am reading this notes and trying to understand how the posterior distribution formulas of the variational variables involved have been calculated. I am confused about the indices of the summation. ...
• 176
1 vote
26 views

### Tied Bayesian Mixture of Gaussians

I am bit confused when it comes to modelling a Bayesian Gaussian mixture model that assumes a shared covariance/precision matrix for all Gaussian components. I followed the derivation in Bishop and ...
• 11
1 vote
232 views

### Why do we start from $KLD(q(z) || p(z|x))$ in ELBO derivation?

Of the several ways to derive the evidence lower bound (such as using Jensen’s inequality), a version often used is the derivation from $KLD(q(z) || p(z|x))$. The following image illustrates the ...
1 vote
89 views

### Is there a reason to use variational inference for point estimates?

I have seen Bayesian hierarchical models, particularly in computational biology, that use variational inference, but do not use the uncertainty provided by a variational solution. For example, MOFA is ...
• 1,140
321 views

### Why Assumed Density Filtering is also called Moment Matching?

I am learning about Assumed Density Filtering (ADF) and Expectation Propagation in the context of bayesian deep neural networks. I have seen in some textbooks and papers that ADF is also called moment ...
145 views

### How to adapt a linear time Newton-Raphson numerical method for an optimisation problem with positivity constraints?

I would appreciate some assistance in understanding how I can adapt a linear time Newton-Raphson root finding algorithm for unconstrained optimisation, to solve a problem where I introduce positivity ...
• 2,206
1 vote
### Is there a conditional version of $\beta$-VAE?
Pretty straightforward question. I could not find any information on the existence of a "conditional $\beta$-VAE". I'm using CVAE for a regression problem and having trouble balancing KL and ...