# Questions tagged [variational-bayes]

Variational Bayesian methods approximate intractable integrals found in Bayesian inference and machine learning. Primarily, these methods serve one of two purposes: Approximating the posterior distribution, or bounding the marginal likelihood of observed data.

243 questions
Filter by
Sorted by
Tagged with
131 views

### ELBO maximization with SGD

In cases such as Gaussian mixture models, there's is no closed-term solution for the original likelihood maximization. Maximizing the ELBO, however, does have analytical update formulas (i.e. formulas ...
16 views

### Mean-field approximation of a bivariate Gaussian

In their overview paper on variational inference (https://arxiv.org/pdf/1601.00670.pdf), Blei et al. show a contour of a two-dimensional Gaussian (Fig. 1, see below) and note that ‘the optimal mean-...
29 views

### Reparameterization trick for exponential distribution

Is there way to generate Exponential(lambda) distributed samples via a reparameterization trick? As in: Reparameterization trick for gamma distribution And also: How does the reparameterization ...
33 views

### Why do variational autoencoders not find the “best” latent variables?

From my understanding: Variational autoencoders sample the latent variables $y$ using a proposal distribution $q$ of the observed variables $x$. The objective is that the decoder $p$ applied to $y$ ...
40 views

### Bishop derivation completing the square in variational inference

I don't understand the derivation on page 467. Bishop says: Given the optimal factor $q_1^*(z_1)$ \begin{equation} ln~q_1(z_1) = -\frac{1}{2} z_1^2 \Lambda_{11} + z_1 \mu_1 \Lambda_{11} - z_1 \...
17 views

### Derivation of the Objective Function for Expectation Propagation

I was reading Expectation Propagation As A Way Of Life and the original paper by Minka Expectation Propagation for Approximate Bayesian Inference and they both say that a fixed point of the EP ...
26 views

43 views

### A technical question about the reparametrisation trick

I was reading this post which enlightened me about the technicalities of the reparametrisation trick, but I only get the intuition of this equivalent transform and I'm not sure why it is true: 𝐸_𝑞[...
54 views

### Is Variational Bayes (VB) and Mean-Field Approximation Useful In practice

I have just had a course in Bayesian Inference, and I am left puzzled about what method should I actually use in practice. Assume I have a multivariate model with multiple parameters $\theta$, where ...
### How to characterize the effect of $(\textrm{Diag}(\Sigma^{-1}))^{-1}$ badly approximating $\textrm{Diag}(\Sigma)$
I have an almost singular covariance matrix $\Sigma\in\mathbb{R}^{n\times n}$ that has a few large eigenvalues, followed by many many comparatively very small ev's. If I were to try to approximate ...