Having watched videos on youtube, I feel like I can't really define what variational inference is. I can follow procedures while I am watching the video lectures about it. But hard to define what really is. Hope to hear about it.
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Please see my answer to a similar question asking about variational inference. To learn more about variational inference, check out the book I've written about the topic. $\endgroup$– Yuri PlotkinCommented Jan 7 at 6:05
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Not based on my knowledge, but here's a paper (in fairly plain English) that I think is very relevant to the question: Blei, Kucukelbir & McAuliffe 2016. Variational Inference: A Review for Statisticians. https://arxiv.org/abs/1601.00670
From the abstract:
One of the core problems of modern statistics is to approximate difficult-to-compute probability densities. This problem is especially important in Bayesian statistics, which frames all inference about unknown quantities as a calculation involving the posterior density. In this paper, we review variational inference (VI), a method from machine learning that approximates probability densities through optimization. VI has been used in many applications and tends to be faster than classical methods, such as Markov chain Monte Carlo sampling. The idea behind VI is to first posit a family of densities and then to find the member of that family which is close to the target. Closeness is measured by Kullback-Leibler divergence. We review the ideas behind mean-field variational inference, discuss the special case of VI applied to exponential family models, present a full example with a Bayesian mixture of Gaussians, and derive a variant that uses stochastic optimization to scale up to massive data. We discuss modern research in VI and highlight important open problems. VI is powerful, but it is not yet well understood. Our hope in writing this paper is to catalyze statistical research on this class of algorithms.
They also offer guidance in when statisticians should use Markov chain Monte Carlo sampling and when variational inference (see paragraph Comparing variational inference and MCMC in the article).
-
1$\begingroup$ I have been reading that paper and it still doesn't make sense to me. Is there somewhere an example with coin flipping or something that can be easily followed? $\endgroup$– thecity2Commented May 10, 2018 at 19:40
-
$\begingroup$ I see this paper is 41 pages; I will read it but I was hoping someone might have a simple example, such as estimating a Beta posterior given binomial success data using VI. A few sources I’ve seen have jumped right into matrix factorization via VI, but I’d really prefer to start with something simpler and more intuitive. $\endgroup$– mjakeCommented Nov 8, 2020 at 18:35