Questions tagged [variational-inference]
The variational-inference tag has no usage guidance.
37
questions
1
vote
2
answers
49
views
Understanding the Evidence Lower Bound (ELBO)
I am reading this tutorial about Variational Inference, which includes the following depiction of ELBO as the lower bound on log-likelihood on the third page. In the tutorial, $x_i$ is the observed ...
- 1,427
0
votes
0
answers
17
views
mean-field variational inference in Nonconjugate Models
I'm trying to conduct mean-field Variational Inference (VI) with Nonconjugate model.
I found Variational Inference in ...
0
votes
0
answers
17
views
Modifying Variational Inference to be robust to outliers?
Normally, for variational inference, you have some evidence data $Z$, you have some true distribution $P(X|Z)$, and you have a simpler parameterized distribution $Q(X|\theta)$, and you're trying to ...
- 389
1
vote
2
answers
54
views
VAE: what activation function (if any) to use for the last layer of my decoder if I don't want to assume any knowledge about the scale of my inputs?
I'm working on an implementation of a Variational Autoencoder (VAE). There are lots of helpful examples and guides out there, which typically introduce VAE in the context of image data, e.g. MNIST. ...
- 133
1
vote
1
answer
23
views
Why do we use the same parameters for the joint, marginal and conditional distributions in VAEs?
I've noticed in several resources on variational autoencoders (for example the Wikipedia article), we use the same parameters theta ($\theta$) for the prior, likelihood, posterior, etc distributions. ...
- 11
3
votes
1
answer
78
views
Why Reparameterization Trick does not work with discrete latent variables?
I came to know from the Youtube Video here (Timestamp 1:03:55) that Reparameterization trick only works for continuous latent variable. But, I am not clear as to why it does not work for discrete ...
- 161
2
votes
2
answers
57
views
Prior in variational autoencoders
I am currently dealing with variational autoencoders where I've read the original paper "An introduction to variational Bayes" from Kingma and Welling. I am currently still a little confused ...
- 21
0
votes
0
answers
15
views
Variational Inference Mean-Field Gaussian
I am new to variational inference and got very confused about some basic ideas. We want to use the mean-field gaussian family to approximate a complicated high-dimensional distribution. I want to ...
- 87
0
votes
0
answers
19
views
Decoder distribution in avariational autoencoder
I have a little perplexity about the variational autoencoder model, by looking at some underlying terminology.
We assume the approximating posterior distribution to be a Gaussian $q_\phi(\textbf{z}|\...
- 437
2
votes
1
answer
24
views
Which exact loss do we minimize in a VAE model?
Reading about VAEs here and there, I often get stuck in the confusion about which quantity gets minimized as VAE objective.
After some calculations, here's what we get at:
$\log p_\theta(x) \ge - \...
- 437
0
votes
0
answers
9
views
Constraining reconstructed vectors to lie in a hyperplane ina VAE
I'm trying to add a linear constraint to my variational autoencoder model. Let's say that my input
is made of two concatenated vectors: $\textbf{x} = \textbf{t} \oplus \textbf{y}$ where (for example) ...
- 437
1
vote
1
answer
46
views
Using a Variational AutoEncoder with an inverse bottleneck
For a problem I'm dealing with, I'm trying to understand if my approach could make sense.
I'm using a Variational AutoEncoder (VAE) having relatively low-dimensional inputs, say $x \in \mathbb{R}^n$. ...
- 437
0
votes
0
answers
93
views
Higher latent space than Input space in a Variational autoencoder (VAE)
Could it make any sense to choose a larger dimension for the latent space of the VAE with respect to the original input?
For example, we may want to learn how to reconstruct a relatively low-...
- 437
0
votes
0
answers
37
views
Why is KL divergence used as a measure of closeness in variational inference?
I am curious why KL divergence is the standard measure of (dis)similarity used in VI while it is not even a proper metric (asymmetric and does not satisfy triangle inequality).
- 223
0
votes
0
answers
26
views
Appendix A on Variational Gaussian Process State Space Model
On Frigola et al
in the Supplementary material A, equation (19) is:
$\prod_{t=1}^{T}p(\mathbf{f}_t|\mathbf{f}_{1:t-1},\mathbf{x}_{0:t-1},\mathbf{u})=\mathcal{N}(\mathbf{f}_{1:T}|\mathbf{K}_{0:T-1,\...
- 101
0
votes
0
answers
23
views
Distribution over parameters vs. distribution over functions
I find it hard to distinguish between these two concepts. In a variational inference setting we learn a distribution over the parameters of our function. in the definition of Gaussian processes we ...
- 157
1
vote
1
answer
56
views
Variational Autoencoder assumtions
I am currently reading the paper "Importance Weighted Autoencoders" and am having a hard time understanding something regarding the original Variational Autoencoder (VAE) as described here
...
- 63
0
votes
0
answers
25
views
Expected value of log(gamma function(Dirichlet variable))
The following problem emerges from coordinate ascent variational inference in a mixture model with Dirichlet-Multinomial components. I want to compute the expectation of the log likelihood. Since my ...
- 699
0
votes
0
answers
10
views
Bayesian Interpretation of Deep Ensembles
I was wondering if training a neural network in the deep ensemble setting can lead to a network with a posterior vs. a point estimate architecture?
Recently there have been discussions over the ...
- 157
0
votes
0
answers
24
views
Best method(s) to estimate the parameters of a stochastic process with a hybrid (i.e. switching) random input variable?
I'm looking for the best approach to and/or methods of solving the following inference problem. I have tried searching for similar questions but don't have enough knowledge on the various methods (...
- 155
3
votes
2
answers
80
views
Derivation of ELBO in ADVI Paper, Jacobian of Elliptical Transformation
I've been following the ELBO derivations in the paper Automatic Differentiation Variational Inference and have a few questions. With the model $p(x,\theta)$, they first transform $\theta$ so that it ...
- 33
0
votes
0
answers
70
views
Variational inference - posterior predictive distribution
So suppose we have a neural network that aims to map values from one distribution to another. That is to say the inputs do not belong to the same distribution as the targets.
It then follows that, ...
- 179
1
vote
1
answer
99
views
Variational Bayes for a univariate Gaussian
I'm following an example from Murphy's book (Sec 21.5.1) on how to apply Variational Bayes to infer the posterior over the parameters for a 1D Gaussian $p(\mu,\lambda|\mathcal{D})$. The example uses a ...
- 643
0
votes
0
answers
16
views
Inhowfar does Variational Inference work better with members of the exponential family?
I am reading Variational Inference: A Review for Statisticians.
Working in [the exponential] family simplifies variational inference: it is easier
to derive the corresponding CAVI algorithm, and it ...
0
votes
0
answers
37
views
Variational inference: Rewriting ELBO
I was following some tutorials on variational inference and ELBO loss functions and I think I understand it quite well, but I'm struggling with math.
If I understand it correctly we went from this ...
- 1
0
votes
0
answers
98
views
Loss Function of a Variational Autoencoder when using Implicit Reparameterization Gradients (Dirichlet distributed latent space)
I would like to implement a VAE with a Dirichlet distributed latent space in Python.
Since the reparametrization trick does not work for the Dirichlet Distribution I would use Implicit ...
0
votes
0
answers
89
views
Variational Autoencoder with Dirichlet distributed latent space using the Weibull Distribution
My goal is to create an VAE with an Dirichlet distributed latent space. Since the reparametrization trick does not work for the Dirichlet Distribution, I am trying to approximate the Gamma ...
0
votes
1
answer
38
views
Can someone explain Stein's method/discrepancy in a way that makes sense?
I have been wanting to understand this paper in a deeper way for a long time Stein Variational Gradient Descent: A General
Purpose Bayesian Inference Algorithm
But everytime I read about Stein's ...
- 629
0
votes
0
answers
43
views
Variational coin tossing from scratch: calculation of the expected log likelihood
I'm working my way through this tutorial about variational inference for a coin tossing. Let's say the probability of the event head is denoted by ...
mc1893
0
votes
1
answer
20
views
Stochatic Variational Bayes with distribution on parameters?
In Autoencoding Variational Bayes, the authors show, in the Appendix F, that SGVB can be performed with a model where we have a distribution over the generative parameters \theta (Full VB). They ...
- 3
0
votes
0
answers
19
views
Derivation of updating rule
I am trying to understand "Variational Bayesian Multiple Instance Learning with Gaussian Processes" by Manuel Haußmann, Fred A. Hamprecht, and Melih Kandemir. Right now I am stuck in the ...
0
votes
0
answers
32
views
Best probability density function to use for the prior of a variance parameter in Bayesian inference
This answer provided some good general advice, but in my specific case I want to create a model of my prior beliefs about the variance of a normally-distributed random variable:
$$x \sim \mathcal{N}(0,...
- 155
2
votes
2
answers
146
views
Inference: How is the Laplace approximation actually useful to us compared with MLE and MAP?
I was reading a few different sources (including the "Machine Learning and Pattern Recognition" book by Bishop) about the Laplace integral approximation method for inference. However, I am ...
0
votes
0
answers
21
views
Differentiating entropy in Reinforcement Learning as Probabilistic Inference
I am studying the paper Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review (https://arxiv.org/abs/1805.00909) and I do not understand how the author differentiate the ...
- 1
0
votes
1
answer
99
views
variational parameters in variational autoencoders
So we have x as the observed variable, and z as the latent variable, denoted by this bayesian network.
And we parameterize x by $\theta$ to get $p_{\theta}(\mathbf{x})$
the posterior is $p_{\theta}(...
- 103
0
votes
0
answers
55
views
Areas of research in statistical machine learning
I'm reading from a book called Machine Learning: A Probabilistic Perspective by Kevin Murphy. Besides being somewhat challenging to understand, I feel that the earlier chapters (on probability, ...
0
votes
0
answers
14
views
Understanding variational inference for Bayesian GMM
I am reading Variational Inference: A Review for Statisticians by Blei, Kucukelbir and McAuliffe. I am having a hard time following some of the steps in Section 3.2
More precisely, they state that $\...
- 793