Skip to main content

Questions tagged [normalizing-flow]

Random variate representations based on repeated one-to-one transforms of a standard random variable which produce generative models and closed-form densities.

Filter by
Sorted by
Tagged with
1 vote
0 answers
11 views

Can normalizing flows approximate bounded distributions in deep learning?

I’m exploring the use of normalizing flows in deep learning for generative modeling and I have a specific requirement: my target distributions are bounded (for example, between 0 and 1). I understand ...
Felipe Vieira's user avatar
0 votes
0 answers
23 views

How to train a continuous-time normalizing flow model?

I'm confused on how we can actually train a continuous-time normalizing flow model. There are two use cases for the discrete-time (original) normalizing flows, and I've tried to outline how I would do ...
travelingbones's user avatar
0 votes
0 answers
38 views

How can we use ReLU activation in a Normalizing Flow model? More generally, is differentiable almost everywhere enough for a normalizing flow?

In some works, e.g., enter link description here normalizing flow models are considered with ReLU activation. For example, using a planar flow, $f = f_n \circ ... \circ f_1$, and each $f_i$ has the ...
travelingbones's user avatar
1 vote
0 answers
11 views

Help finding Bayesian model with multi-modal posterior [closed]

Background: There is a paper (link) that concerns combining MCMC methods with a normalizing flow (a type of generative model). The basic idea is that the normalizing flow helps propose samples, which ...
caitlin's user avatar
  • 31
1 vote
0 answers
23 views

At what circumstances will the difficulty for the tasks of density evaluation and sampling be different?

In this tutorial video of normalizing flow, the presenter mentioned that for the original autoregressive flow, the density evaluation is fast and the sampling is slow. In contrast, for the inverse ...
8cold8hot's user avatar
  • 141
5 votes
0 answers
145 views

What is the difference between copulas and normalizing flows?

The goal of normalizing flows is to produce arbitrarily complex probability-distributions from a simple distribution (usually the Normal distribution) through learning an invertible transform. Copulas ...
smthack's user avatar
  • 61
1 vote
0 answers
53 views

Choice of base distribution in normalizing flows for product distribution

I'm currently trying to implement a normalizing flow (NF) to efficiently sample from a product distribution $p=fg$. Contrary to most examples I've come across, I actually know how to sample ...
Hubble's user avatar
  • 131
0 votes
1 answer
90 views

Normalizing Flows Invertibility

I am currently reading up on RealNVP, which has the following transformations according Lilian Weng: $$ \begin{aligned} \mathbf{y}_{1:d} &= \mathbf{x}_{1:d} \\ \mathbf{y}_{d+1:D} &= \mathbf{x}...
user2793618's user avatar
0 votes
0 answers
77 views

Normalizing Flow Penalization

I am looking to train a normalizing flow, specifically a Masked Autoregressive Flow model. However, this model leads to high variance on lower dimensional, less complex data. I am using a neural ...
user2793618's user avatar
4 votes
2 answers
1k views

Normalizing Flows KL divergence equivalency

This question is related to the normalizing flows concept in machine learning. Let $X \sim P_X$ and $U \sim P_U$ be, respectively, the distribution of the data and a base distribution (e.g. an ...
Chen's user avatar
  • 141
2 votes
1 answer
154 views

Which parameters are updated in VAE with normalizing flow?

I've been reading this article about implementing a VAE with normalizing flows. What it's not clear to me, is which parameters are actually optimized using this approach. Should I only optimize the ...
James Arten's user avatar
0 votes
1 answer
67 views

Planar flows implementation to approximate Gamma distribution

I've been trying to implement in order to approximate Gamma distribution but the problem I've been encountering is that the hyperbolic tangent activation function that I used gives negative values, ...
Aidana Abdikarim's user avatar
5 votes
1 answer
285 views

loss function that penalizes empirical CDF

I have been doing literature review of generative models. From what I gather, there are likelihood based generative models that model the likelihood and use it as objective function to learn the ...
secondrate's user avatar
2 votes
1 answer
268 views

Issues with GAN and VAE models

I'm reading this amazing paper on Normalizing Flows https://arxiv.org/pdf/1908.09257.pdf but one sentence kind of bothers me: GANs and VAEs have demonstrated impressive performance results on ...
Pavel Podlipensky's user avatar
1 vote
0 answers
116 views

Neural Networks with Tractable Integral [closed]

I'm looking for neural networks $n_\theta(x)$ which integrate to a constant $\int_{[0,1]^d} n(x)\; dx=c\in\mathbb{R}$. Notably, the constant should be the same for any weights $\theta$ for which $n_\...
Alexander Mathiasen's user avatar
4 votes
2 answers
390 views

Why aren't Normalizing Flows suitable for Discrete Distributions?

I am currently trying to understand why normalizing flows are not applicable to discrete distributions (a quick primer on NF can be found here). The assumptions on the transformation f between the ...
N8_Coder's user avatar
  • 219
0 votes
1 answer
112 views

Proof for expressive power of flow-based models: Normalizing Flows for Probabilistic Modeling and Inference by Papamakarios et al

I've been reading the great summary work on Normalizing Flows "Normalizing Flows for Probabilistic Modeling and Inference" by Papamakarios et al.. A few questions regarding a proof came up ...
Jan Leger's user avatar
0 votes
0 answers
111 views

Inverse Neural Networks

Suppose there is a series of transformation applied to the random variable $z_0$ such that $$ z_M = f_{\theta_{M}} \circ f_{\theta_{M-1}} \circ \ldots \circ f_{\theta_{1}}(z_0) =: f_{\theta}(z_0). $$ ...
Josh Pilipovsky's user avatar
5 votes
1 answer
2k views

Normalizing flows as a generalization of variational autoencoders?

Normalizing flows are often introduced as a way of restricting the rigid priors that are placed on the latent variables in Variational Autoencoders. For example, from the Pyro docs: In standard ...
ostrokach's user avatar
  • 299
3 votes
1 answer
2k views

Normalizing flow training

I've been learning about normalizing flow. This is my understanding, and please correct me whenever I am wrong. There are $\{y_1,y_2,...,y_n\}$ samples from an unknown distribution $p_y(y)$ that we ...
Schach21's user avatar
  • 145
1 vote
0 answers
86 views

Normalizing Flows notations and change of variable formula

Maybe it is a silly question, but I am wondering why in some papers the function $f$ is the mapping from $x$ to $z$ and in some others it is the other way around. Typically in this review we go from $...
glouis's user avatar
  • 227
1 vote
1 answer
803 views

What is multi-scale architecture?

When I was reading the paper "Density Estimation using Real NVP", I have found the term "multi scale architecture". We implement a multi-scale architecture using a squeezing ...
user27286's user avatar
  • 299
4 votes
1 answer
979 views

Inference in Normalizing Flow model: NICE(non linear independent components estimation)

I was recently reading Y. Bengio's paper on NICE (https://arxiv.org/abs/1410.8516). In the paper, authors have taken a view that a good representation involves easy learning of the data distribution. ...
Lawhatre's user avatar
  • 171
4 votes
1 answer
790 views

Vector-Jacobian Product Computational Cost

The paper FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models presents a continuous-time flow as a generative model which uses Hutchinson's trace estimator to give an ...
Lashoun's user avatar
  • 43
1 vote
0 answers
38 views

Can we ignore the generation side of the method described in density estimation using Real NVP?

First appologies if my question is stupid. I am studying the paper "Density estimation using real NVP" by Dinh, Sohl-Dickstein and Bengio. link The paper presented a nice idea that the generation ...
honglei's user avatar
  • 11
7 votes
2 answers
3k views

Planar Flow in Normalizing Flows

While I've read "Variational Inference with Normalizing Flows" (abstract), I don't understand about an intuition of Planar Flow. The author defined Planar Flow as below Let $\boldsymbol{w} \in \...
alryosha's user avatar
  • 271
6 votes
2 answers
5k views

Difference between invertible NN and flow-based NN

After a little bit of reading on these two terms, I have the impression they are used for the same thing. So is there actually a difference between these two concepts, and if so, how are they ...
flawr's user avatar
  • 583
7 votes
1 answer
2k views

What are the advantages of normalizing flow over VAEs with deep latent gaussian models for inference?

I am reading the normalizing flow paper and am a bit confused. It seems that being able to model complex (correlated?) posterior is one of the advantages of the proposed approach (Section 2.3, last ...
CP Tai's user avatar
  • 115