Questions tagged [normalizing-flow]
Random variate representations based on repeated one-to-one transforms of a standard random variable which produce generative models and closed-form densities.
23
questions
3
votes
0
answers
56
views
What is the difference between copulas and normalizing flows?
The goal of normalizing flows is to produce arbitrarily complex probability-distributions from a simple distribution (usually the Normal distribution) through learning an invertible transform.
Copulas ...
- 74.6k
1
vote
0
answers
33
views
Choice of base distribution in normalizing flows for product distribution
I'm currently trying to implement a normalizing flow (NF) to efficiently sample from a product distribution $p=fg$. Contrary to most examples I've come across, I actually know how to sample ...
- 131
0
votes
1
answer
57
views
Normalizing Flows Invertibility
I am currently reading up on RealNVP, which has the following transformations according Lilian Weng:
$$
\begin{aligned}
\mathbf{y}_{1:d} &= \mathbf{x}_{1:d} \\
\mathbf{y}_{d+1:D} &= \mathbf{x}...
- 36
4
votes
2
answers
669
views
Normalizing Flows KL divergence equivalency
This question is related to the normalizing flows concept in machine learning.
Let $X \sim P_X$ and $U \sim P_U$ be, respectively, the distribution of the data and a base distribution (e.g. an ...
- 71.8k
0
votes
1
answer
55
views
Planar flows implementation to approximate Gamma distribution
I've been trying to implement in order to approximate Gamma distribution but the problem I've been encountering is that the hyperbolic tangent activation function that I used gives negative values, ...
0
votes
0
answers
70
views
Normalizing Flow Penalization
I am looking to train a normalizing flow, specifically a Masked Autoregressive Flow model. However, this model leads to high variance on lower dimensional, less complex data. I am using a neural ...
- 19
2
votes
1
answer
123
views
Which parameters are updated in VAE with normalizing flow?
I've been reading this article about implementing a VAE with normalizing flows. What it's not clear to me, is which parameters are actually optimized using this approach. Should I only optimize the ...
- 8,397
3
votes
1
answer
322
views
Why aren't Normalizing Flows suitable for Discrete Distributions?
I am currently trying to understand why normalizing flows are not applicable to discrete distributions (a quick primer on NF can be found here).
The assumptions on the transformation f between the ...
- 46
4
votes
1
answer
235
views
loss function that penalizes empirical CDF
I have been doing literature review of generative models. From what I gather, there are likelihood based generative models that model the likelihood and use it as objective function to learn the ...
- 25.6k
2
votes
1
answer
242
views
Issues with GAN and VAE models
I'm reading this amazing paper on Normalizing Flows https://arxiv.org/pdf/1908.09257.pdf but one sentence kind of bothers me:
GANs and VAEs have demonstrated impressive performance results on
...
- 56.3k
6
votes
2
answers
5k
views
Difference between invertible NN and flow-based NN
After a little bit of reading on these two terms, I have the impression they are used for the same thing. So is there actually a difference between these two concepts, and if so, how are they ...
- 74.6k
1
vote
0
answers
105
views
Neural Networks with Tractable Integral [closed]
I'm looking for neural networks $n_\theta(x)$ which integrate to a constant $\int_{[0,1]^d} n(x)\; dx=c\in\mathbb{R}$. Notably, the constant should be the same for any weights $\theta$ for which $n_\...
0
votes
0
answers
99
views
Proof for expressive power of flow-based models: Normalizing Flows for Probabilistic Modeling and Inference by Papamakarios et al
I've ben reading the great summary work on Normalising Flows "Normalizing Flows for Probabilistic Modeling and Inference" by Papamakarios et al..
A few questions regarding a proof came up as ...
0
votes
0
answers
109
views
Inverse Neural Networks
Suppose there is a series of transformation applied to the random variable $z_0$ such that
$$
z_M = f_{\theta_{M}} \circ f_{\theta_{M-1}} \circ \ldots \circ f_{\theta_{1}}(z_0) =: f_{\theta}(z_0).
$$
...
- 125
3
votes
1
answer
667
views
Vector-Jacobian Product Computational Cost
The paper FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models presents a continuous-time flow as a generative model which uses Hutchinson's trace estimator to give an ...
- 208
3
votes
1
answer
1k
views
Normalizing flow training
I've been learning about normalizing flow. This is my understanding, and please correct me whenever I am wrong.
There are $\{y_1,y_2,...,y_n\}$ samples from an unknown distribution $p_y(y)$ that we ...
- 4,912
3
votes
0
answers
1k
views
Normalizing flows as a generalization of variational autoencoders?
Normalizing flows are often introduced as a way of restricting the rigid priors that are placed on the latent variables in Variational Autoencoders.
For example, from the Pyro docs:
In standard ...
- 8,397
1
vote
0
answers
76
views
Normalizing Flows notations and change of variable formula
Maybe it is a silly question, but I am wondering why in some papers the function $f$ is the mapping from $x$ to $z$ and in some others it is the other way around. Typically in this review we go from $...
- 8,397
4
votes
1
answer
833
views
Inference in Normalizing Flow model: NICE(non linear independent components estimation)
I was recently reading Y. Bengio's paper on NICE (https://arxiv.org/abs/1410.8516). In the paper, authors have taken a view that a good representation involves easy learning of the data distribution. ...
- 25.6k
1
vote
1
answer
704
views
What is multi-scale architecture?
When I was reading the paper "Density Estimation using Real NVP", I have found the term "multi scale architecture".
We implement a multi-scale architecture using a squeezing ...
- 25.6k
5
votes
2
answers
2k
views
Planar Flow in Normalizing Flows
While I've read "Variational Inference with Normalizing Flows" (abstract), I don't understand about an intuition of Planar Flow.
The author defined Planar Flow as below
Let $\boldsymbol{w} \in \...
- 316k
1
vote
0
answers
36
views
Can we ignore the generation side of the method described in density estimation using Real NVP?
First appologies if my question is stupid. I am studying the paper "Density estimation using real NVP" by Dinh, Sohl-Dickstein and Bengio. link
The paper presented a nice idea that the generation ...
- 11
7
votes
1
answer
2k
views
What are the advantages of normalizing flow over VAEs with deep latent gaussian models for inference?
I am reading the normalizing flow paper and am a bit confused.
It seems that being able to model complex (correlated?) posterior is one of the advantages of the proposed approach (Section 2.3, last ...
- 18.3k