# Questions tagged [generative-models]

A probabilistic or statistical model thought about as describing how the values in a sample is actually generated, and not only as a description or approximation.

303 questions
Filter by
Sorted by
Tagged with
103k views

### Generative vs. discriminative

I know that generative means "based on $P(x,y)$" and discriminative means "based on $P(y|x)$," but I'm confused on several points: Wikipedia (+ many other hits on the web) classify things like SVMs ...
• 3,191
101k views

### How does the reparameterization trick for VAEs work and why is it important?

How does the reparameterization trick for variational autoencoders (VAE) work? Is there an intuitive and easy explanation without simplifying the underlying math? And why do we need the 'trick'?
• 2,834
6k views

### Why in Variational Auto Encoder (Gaussian variational family) we model $\log\sigma^2$ and not $\sigma^2$ (or $\sigma$) itself?

In theory the encoder in VAE (assuming that variational family is Gaussian) generates the $\mu$ and $\sigma$ (or $\sigma^2$). But, in practice, I have seen people assuming the output is $\log\sigma^2$....
• 1,502
8k views

### Likelihood-free inference - what does it mean?

Recently I have become aware of 'likelihood-free' methods being bandied about in literature. However I am not clear on what it means for an inference or optimization method to be likelihood-free. In ...
• 947
6k views

### Modern Use Cases of Restricted Boltzmann Machines (RBM's)?

Background: A lot of the modern research in the past ~4 years (post alexnet) seems to have moved away from using generative pretraining for neural networks to achieve state of the art classification ...
• 615
5k views

### The connection between Bayesian statistics and generative modeling

Can someone refer me to a good reference that explains the connection between Bayesian statistics and generative modeling techniques? Why do we usually use generative models with Bayesian techniques? ...
• 889
18k views

### Variational Autoencoder − Dimension of the latent space

I've done some experiments to understand the influence of the dimension of the latent space in a VAE, and it seems that the higher the space, the harder it is to generate realistic images. I might ...
• 263
5k views

### Why are Gaussian "discriminant" analysis models called so?

Gaussian discriminant analysis models learn $P(x|y)$ and then apply Bayes rule to evaluate $$P(y|x) = \frac{P(x|y)P_{prior}(y)}{\Sigma_{g \in Y} P(x|g) P_{prior}(g) }.$$ Hence, they are generative ...
• 2,402
2k views

### Is the optimization of the Gaussian VAE well-posed?

In a Variational Autoencoder (VAE), given some data $x$ and latent variables $t$ with prior distribution $p(t) = \mathcal{N}(t \mid 0, I)$, the encoder aims to learn a distribution $q_{\phi}(t)$ that ...
• 707
7k views

### What are the current methods to check for GAN overfitting?

In generative modeling, the goal is to find a way for a model to output samples of some distribution $p_X$ given a lot of samples $x_1, \ldots, x_n$. In particular, we want sampling from our model $G$ ...
• 627
712 views

### How to understand Generative Adversarial Networks Discriminative distribution?

So I am currently studying Generative Adversarial Network and I read the paper by Goodfellow a few times now Generative Adversarial Nets and a few other papers in this field (DCGAN, CycleGAN, pix2pix, ...
• 235
18k views

### Why use Binary Cross Entropy for Generator in Adversarial Networks

I'm trying to work with General Adversarial Networks and there's something I'm seeing everywhere but can't explain why... the GANs are usually constructed from a Generator (which usually generates an ...
• 515
2k views

• 942
1k views

### How well does $Q(z|X)$ match $N(0,I)$ in variational autoencoders?

At train time, the KL divergence term drives $Q(z=\mu(X)+\epsilon \times\Sigma(X) | X)$ toward $N(0,I)$, where $\epsilon\sim N(0,I)$. It can't drive $Q(z|X)$ to exactly $N(0,I)$ because the ...
• 253
2k views

### Why aren't auto-encoders also considered generative models?

Auto-encoders (AEs) are composed of an encoder and a decoder (often represented by a neural network). The encoder produces a vector representation $z$ of its input $x$ (e.g. an image). The decoder ...
4k views

### Generative models for time series simulation

I have a basic understanding of generative models, like Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs). It seems that they are mostly used to in the field of image ...
• 565
200 views

### Structure of Generative Adversarial Networks (GAN) for mapping a simulation model

There is a simulation model of a system that I want to map as a neural network to test if a better execution time can be achieved with similar accuracy. The simulation model receives real-valued ...
• 93
1k views

### Clarification: Are Generative Adversarial Networks an alternative to MCMC sampling?

I have been reading the original Goodfellow, et. al. paper on Generative Adversarial Networks and the way that they can obtain estimates of the posterior distribution of a discriminative network or ...
• 1,522
4k views

### Wasserstein Loss is very sensitive to model architecture

I am working on a class project where I compare the performance of GAN and WGAN. Since the only difference between GAN and WGAN is the Wasserstein loss, I chose one neural network model architecture ...
• 193
1k views

### reparameterization trick in VAEs, How should we do this?

I'm confused about how does reparameterization trick works. In this article shows it very simple. You learn two vectors $\sigma$ and $\mu$, sample $\epsilon$ from $N(0, 1)$ and then your latent ...
• 309
2k views

### Linking generative, discriminative models to supervised and unsupervised learning

Definitions that I am considering: A generative model learns p(x,y) whereas a discriminative model learns p(y|x=x). I would like to verify if my understanding is correct by sharing the following ...
4k views

### How well does GAN (generative adversarial network) perform for small samples?

GAN is an unsupervised learning algorithm that pits a discriminator and generator against one another so that they iteratively compete to enhance the overall model's ability to model/replicate a given ...
• 4,005
3k views

• 4,067
4k views

### Why we learn $\log{\sigma^2}$ in VAE reparameterization trick instead of standard deviation? [duplicate]

We know that the reparameterization trick is to learn two vectors $\sigma$ and $\mu$, sample $\epsilon$ from $N(0, 1)$ and then your latent vector $Z$ would be (where $\odot$ is the element-wise ...
• 309
4k views

### How do you evaluate a generative model?

Evaluating a discriminative model is relatively easy: compare the predictions with ground truth, using cross-validation. Unfortunately this strategy can't be used for generative models. Surely this ...
• 775
6k views

### How to interpret the following GAN training losses?

I am training a GAN using the following loss functions: ...
• 181
1k views

### What can possibly go wrong in a Generative Adversarial Network?

Lately, after reading about GANs, I started experimenting with the MNIST dataset, and the result we acceptable. Here are some details about the networks I used: Discriminator: 784 inputs $\rightarrow$...
• 223
1k views

### How do you calculate log likelihood p(x) for a VAE?

I was reading the Importance Weighted Autoencoders paper and its experiment section compares the density estimation result on MNIST for IWAE vs VAE. I know that density estimation estimating log p(x) ...
• 139
404 views

### What methods can be used for distribution generation other than GANs?

Generative Adversarial Networks (GANs) can be used for creating distributions of data points, that follow source data set distributions (e.g. images, sound, text, etc). Are there any other methods or ...
• 153
945 views

### What is the correct definition of the Likelihood function?

I am doing the CS229:Machine Learning of Stanford Engineering Everywhere. All trhough the first chapter he uses $$L(\theta) = P(Y | X; \theta)$$ i.e. the likelihood of the parameter $\theta$ is ...
• 265
237 views

### Why do we care if the likelihood function is tractable?

I'm learning (deep) generative models and I've seen many places where the difficulty is that the likelihood function (with some parameters defined by our model) is intractable, i.e., it involves ...
2k views

### Why does enforcing a prior distribution create semantic latent variables in variational autoencoders?

Variational autoencoders create latent variables that have a known distribution (e.g., Gaussian with zero mean and unit variance), and so do adversarial autoencoders. I understand why this turns the ...
• 720
2k views

### Restricted Boltzmann Machines vs GAN

Can someone please tell me how RBMs and GANs compare to each other? I know that the community is more excited over GANs than RBMs at the moment. I guess it's because GANs produce better results? My ...
• 225
210 views

### Is generating training data for a classifier mathematically valid?

I have an application with a class imbalance problem: A lot of positive data points, but very few (comparatively) negative points. One colleague recommended that I train a generative model on the ...
• 2,872
354 views

### Tutorial recommendations for understanding GANs

I understand that generative adversarial networks (GANs) can synthetically reconstruct the input using a generator and a discriminator in a zero-sum game. However, I feel that I do not fully ...
938 views

### What is the relationship between generative models and density estimation?

If aren't they synonymous, what distinguishes the one from the other? Is probability density estimation a certain kind of generative model? Can any generative model be regarded as density estimation?
• 1,063
117 views

### Convergence to gradient in limit of variance

I came across this equation in the original GAN paper (pg 2 https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf): \lim_{\sigma \rightarrow 0} \nabla_{\bf x} \mathbb{E}_{\epsilon \sim \...
• 205