Questions tagged [normalizing-flow]

Random variate representations based on repeated one-to-one transforms of a standard random variable which produce generative models and closed-form densities.

Filter by
Sorted by
Tagged with
0 votes
0 answers
20 views

Can we train normalizing flows with Wasserstein distance?

To train flow based models, you usually either use forward or reverse kl as your loss function. My question is, can you use wasserstein distance directly as your loss function to replace kl? I have ...
user avatar
  • 101
3 votes
1 answer
50 views

Normalizing Flows KL divergence equivalency

This question is related to the normalizing flows concept in machine learning. Let $X \sim P_X$ and $U \sim P_U$ be, respectively, the distribution of the data and a base distribution (e.g. an ...
user avatar
  • 131
1 vote
1 answer
23 views

Which parameters are updated in VAE with normalizing flow?

I've been reading this article about implementing a VAE with normalizing flows. What it's not clear to me, is which parameters are actually optimized using this approach. Should I only optimize the ...
user avatar
0 votes
0 answers
19 views

Forcing normalizing flow to output only positive values

What practical tricks used for training planar normalizing flow if a target distribution defined only for positive values? I've considered a toy example with gamma distribution. The first thing that ...
user avatar
0 votes
0 answers
19 views

Planar flows implementation to approximate Gamma distribution

I've been trying to implement in order to approximate Gamma distribution but the problem I've been encountering is that the hyperbolic tangent activation function that I used gives negative values, ...
user avatar
3 votes
1 answer
97 views

loss function that penalizes empirical CDF

I have been doing literature review of generative models. From what I gather, there are likelihood based generative models that model the likelihood and use it as objective function to learn the ...
user avatar
0 votes
0 answers
33 views

Deriving Posterior with Nomalizing Flows

Typically, a Normal distribution is a conjugate prior for $\mu$ of a Normal distribution, we have a closed-form solution to update realize the Bayesian update. For example in Bayesian linear ...
user avatar
1 vote
1 answer
94 views

Issues with GAN and VAE models

I'm reading this amazing paper on Normalizing Flows https://arxiv.org/pdf/1908.09257.pdf but one sentence kind of bothers me: GANs and VAEs have demonstrated impressive performance results on ...
user avatar
1 vote
0 answers
30 views

Neural Networks with Tractable Integral [closed]

I'm looking for neural networks $n_\theta(x)$ which integrate to a constant $\int_{[0,1]^d} n(x)\; dx=c\in\mathbb{R}$. Notably, the constant should be the same for any weights $\theta$ for which $n_\...
user avatar
0 votes
0 answers
44 views

Anomaly detection given conditional log likelihood

I have a trained generative model (normalizing flow) that gives me conditional log-likelihoods of any point $x_n$ given a set of points $y={y_0,y_1...}$. My goal is to do anomaly detection but it is ...
user avatar
0 votes
1 answer
97 views

Why aren't Normalizing Flows suitable for Discrete Distributions?

I am currently trying to understand why normalizing flows are not applicable to discrete distributions (a quick primer on NF can be found here). The assumptions on the transformation f between the ...
user avatar
  • 123
0 votes
0 answers
49 views

Proof for expressive power of flow-based models: Normalizing Flows for Probabilistic Modeling and Inference by Papamakarios et al

I've ben reading the great summary work on Normalising Flows "Normalizing Flows for Probabilistic Modeling and Inference" by Papamakarios et al.. A few questions regarding a proof came up as ...
user avatar
0 votes
0 answers
60 views

Inverse Neural Networks

Suppose there is a series of transformation applied to the random variable $z_0$ such that $$ z_M = f_{\theta_{M}} \circ f_{\theta_{M-1}} \circ \ldots \circ f_{\theta_{1}}(z_0) =: f_{\theta}(z_0). $$ ...
user avatar
0 votes
0 answers
74 views

Normalizing Flows Applied to Dynamical Systems

Suppose $x : [t_0,t_f] \rightarrow \mathbb{R}^{n}$ is a trajectory that satisfies the linear ODE $$ \dot{x} = Ax, \quad x_0 \sim \rho_0, $$ where $\rho_0$ is the PDF of the initial state $x_0$. Thus, ...
user avatar
1 vote
0 answers
167 views

Normalizing flows as a generalization of variational autoencoders?

Normalizing flows are often introduced as a way of restricting the rigid priors that are placed on the latent variables in Variational Autoencoders. For example, from the Pyro docs: In standard ...
user avatar
  • 259
2 votes
1 answer
470 views

Normalizing flow training

I've been learning about normalizing flow. This is my understanding, and please correct me whenever I am wrong. There are $\{y_1,y_2,...,y_n\}$ samples from an unknown distribution $p_y(y)$ that we ...
user avatar
  • 155
1 vote
0 answers
41 views

Normalizing Flows notations and change of variable formula

Maybe it is a silly question, but I am wondering why in some papers the function $f$ is the mapping from $x$ to $z$ and in some others it is the other way around. Typically in this review we go from $...
user avatar
  • 215
1 vote
1 answer
175 views

What is multi-scale architecture?

When I was reading the paper "Density Estimation using Real NVP", I have found the term "multi scale architecture". We implement a multi-scale architecture using a squeezing ...
user avatar
  • 279
2 votes
1 answer
270 views

Inference in Normalizing Flow model: NICE(non linear independent components estimation)

I was recently reading Y. Bengio's paper on NICE (https://arxiv.org/abs/1410.8516). In the paper, authors have taken a view that a good representation involves easy learning of the data distribution. ...
user avatar
  • 101
3 votes
1 answer
214 views

Vector-Jacobian Product Computational Cost

The paper FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models presents a continuous-time flow as a generative model which uses Hutchinson's trace estimator to give an ...
user avatar
  • 31
1 vote
0 answers
25 views

Can we ignore the generation side of the method described in density estimation using Real NVP?

First appologies if my question is stupid. I am studying the paper "Density estimation using real NVP" by Dinh, Sohl-Dickstein and Bengio. link The paper presented a nice idea that the generation ...
user avatar
  • 11
3 votes
2 answers
1k views

Planar Flow in Normalizing Flows

While I've read "Variational Inference with Normalizing Flows" (abstract), I don't understand about an intuition of Planar Flow. The author defined Planar Flow as below Let $\boldsymbol{w} \in \...
user avatar
  • 199
5 votes
2 answers
3k views

Difference between invertible NN and flow-based NN

After a little bit of reading on these two terms, I have the impression they are used for the same thing. So is there actually a difference between these two concepts, and if so, how are they ...
user avatar
  • 563
6 votes
1 answer
2k views

What are the advantages of normalizing flow over VAEs with deep latent gaussian models for inference?

I am reading the normalizing flow paper and am a bit confused. It seems that being able to model complex (correlated?) posterior is one of the advantages of the proposed approach (Section 2.3, last ...
user avatar
  • 105