# Questions tagged [variational-bayes]

Variational Bayesian methods approximate intractable integrals found in Bayesian inference and machine learning. Primarily, these methods serve one of two purposes: Approximating the posterior distribution, or bounding the marginal likelihood of observed data.

239 questions
Filter by
Sorted by
Tagged with
250 views

### Why computing P(x,D) is simpler than P(x|D) in exponential bayesian networks?

I am reading this tutorial on variational inference and wonder why the statement in the question title which is mentioned on page 3 is true.
378 views

### How can variational inference (for LDA) be explained in layman's terms?

I am learning probabilistic topic modeling, and I am studying latent Dirichlet allocation, specifically the inference process. I found many mathematical details, but I need a layman's explanation and/...
84 views

### Model comparison using lower bound from variational approximation

I applied variational approximation for probit regression model and got the lower bound for the log marginal likelihood. When I compare models with different covariates using lower bound, I found that ...
656 views

### choosing prior parameters for variational mixture of Gaussians

I am implementing a vanilla variational mixture of multivariate Gaussians, as per Chapter 10 of Pattern Recognition and Machine Learning (Bishop, 2007). The Bayesian approach requires to specify (...
455 views

### Why do we use the mean-field approximation for variational Bayes?

I often see the mean-field approximation for Variational Bayes. I understand the independence assumption: what I don't understand is why we make that assumption. How does it help us?
150 views

33 views

### How extract an appropriate mean from a posterior distribution

I am studying deterministic inference algorithms to learn posterior probability of Gaussian distributions and we need to find the hyperparameters for the mean and variance random variables of the ...
82 views

### how does one compute expectations for non-linear functions

I am continuing my struggles with approximate Bayesian inference methods. I have a fundamental doubt about how to compute certain expectations that arise during variational bayes, for example. So, my ...
75 views

884 views

### Variational Bayes: Understanding Mean field approximation

I am looking at the mean field approximation as used in Variational Bayes inference and I looked at this section on wikipedia with the factorised approximation as described here: https://en.wikipedia....
218 views

### Variational Bayes combined with Monte Carlo

I'm reading up on variational Bayes, and as I understand it, it comes down to the idea that you approximate $p(z\mid x)$ (where $z$ are the latent variables of your model and $x$ the observed data) ...
2k views

### Topic modeling (LDA) gives different outputs

I am using Topic Modeling Tool which is based on Mallet and using latent dirichlet allocation (LDA). When I ran the tool multiple times, with the same input (a folder of 200-500 short text files), and ...
2k views

### variational inference with KL

i am self-studying variational inference - and in Murphy's book "A probabilistic perspective on machine learning" it is discussed that minimizing the forward KL divergence (which is stated to be zero-...
177 views

444 views

### Mean field variational inference

In Chris Bishop PRML book p.465 equation 10.6, the derivation doesn't explain why exactly the term $\int q_j ln(q_j) dz_j$ was generated, is not that term supposed to be multiplied by constant, did ...
542 views

### Reparameterization of probability distribution (spike and slab)

I try to understand a statement in this paper: http://papers.nips.cc/paper/4305-spike-and-slab-variational-inference-for-multi-task-and-multiple-kernel-learning.pdf In particular, I am talking about ...
213 views

### Find expectation or lower bound of log erf

I need to find the expectation of $\log \Phi(x)=\log \left(\int_{-\infty}^x\frac{1}{2\pi}\exp(-\frac{1}{2}s^2)ds\right)$. (I realise this isn't quite the error function, but not sure what to call it). ...
222 views

### Expectation-Maximization with dependent latent variables

Deriving the equations for a Expectation Maximization over my model, I end up with a posterior for the latent variables (E-step) that prevents me from going on. Generative model I assume my data is ...
137 views

### Backward message passing in variational Bayesian inference

I have come across in a research paper that, I do understand the logic. But the paper has't mentioned about the way of updating $\eta_{t}$. When I asked from the authors they said when we equate the ...
263 views

577 views

6k views

### Relation between variational Bayes and EM

I read somewhere that Variational Bayes method is a generalization of the EM algorithm. Indeed, the iterative parts of the algorithms are very similar. In order to test whether the EM algorithm is a ...