# Questions tagged [generative-models]

A probabilistic or statistical model thought about as describing how the values in a sample is actually generated, and not only as a description or approximation.

248 questions
Filter by
Sorted by
Tagged with
9 views

### What is the learning procedure and purpose of class embeddings in Conditional GAN's

I was learning about conditional GAN's, and there is a point I did not understand about the training process. To specify the class we want to generate images from, we specify a number, which is then ...
• 135
11 views

### What is the exact role of model $p_\theta$ in Diffusion models for the reverse process?

I'm reading this interesting blog post explaining Diffusion probabilistic models and trying to understand the following. In order to compute the reverse process, we need to consider the posterior ...
• 437
640 views

### Why Logistic Regression is not a generative model?

I was reading about the difference between discriminative and generative models, and I read that Discriminative models learn only the boundary between classes hence they are not able to to create new ...
• 135
1 vote
35 views

### How to reconstruct a Euclidean distance matrix from grouped pairwise-distance means and standard deviations?

Coordinates and Labels Take the simple case of 3 distinct object classes and 5 instances of each class situated in 3D Euclidean space. The coordinates and labels might look like the following: ...
• 69
1 vote
7 views

### How does the fixed point interation in invertible resnets work?

I feel like I am missing some easy point about this invertible resnet paper which is making it hard for me to grasp how the fixed point iteration works. stated simply, the residual connection in a ...
• 629
1 vote
61 views

### Classical VAE not learning 2D gaussian mixture distribution using MSE loss

I've been exploring VAE for non-image data. I consider small to medium-sized continuous vector spaces and I want to learn the distribution of a dataset in that space. As a warm up exercise, I tried ...
• 113
1 vote
23 views

### Why do we use the same parameters for the joint, marginal and conditional distributions in VAEs?

I've noticed in several resources on variational autoencoders (for example the Wikipedia article), we use the same parameters theta ($\theta$) for the prior, likelihood, posterior, etc distributions. ...
• 11
21 views

### Techniques Used for DeepFake and Its Corresponding Research Field

I'm a beginner in image generative models, I'm trying to do some work similar to DeepFake, therefore I would like to find out first what techniques DeepFake use to generate the fake videos. Do they ...
• 1,663
1 vote
39 views

### Does a CNN always learn a latent space?

In general, a latent space is a structure of reduced dimensionality than that of the input space where points on this space share resemblance the closer they are to each other. This article also ...
• 649
15 views

### Using Inception and FID scores in training?

Is it possible to use the Inception and FID scores in the training of a deep image generation model, i.e. to maximize the scores in a loss function, albeit this is "cheating"? If so, has ...
1 vote
33 views

### GANs - why does the generator want to minimize the loss (intuition)

I am a little bit puzzled about the following. In a generative adversarial network, we consider a binary classification problem with a binary cross-entropy loss. Now, the generator wants to minimize ...
• 111
91 views

### Should I be using batchnorm and/or dropout in a VAE or GAN?

I am trying to design some generative NN models on datasets of RGB images and was debating on whether I should be using dropout and/or batch norm. Here are my thoughts (I may be completely wrong): ...
1 vote
54 views

### Initialization of GAN discriminator

The question is pretty straightforward: how are GAN and WGAN discriminators typically initialized? I couldn't find much info on this. E.x. for GANs, I imagine you would theoretically want the ...
• 133
52 views

### Restricted Boltzmann Machine: W matrix visualization results after training MNIST images and Pseudo-log-likelihood

I am implementing RBM from scratch using Tensorflow and after training my RBM on the MNIST dataset for 200 epochs using Persistent CD with two steps of contrastive divergence, I learn the weights W ...
• 21
1 vote
44 views

### How to estimate joint probability or conditional probability using marginal ones?

I have 2 datasets: The 1st one gives us the probability that $m$ events occur on $n$ observations ($m$ columns for $n$ rows) The 2nd one tells us if the event occurred (1 if occurred, 0 else ; always ...
33 views

### A Generative model for binary classification (Modeling, Network architecture

I'm trying to build a network to classify input $X$ into 2 categories with a generative manner. This is because there is harsh imbalance issue in data. But I can't understand how to train the model. ...
• 1
11 views

### Problem for training Wasserstein GAN

I'm trying to train a Wasserstein GAN to guess sparse one-hot encoded matrices (0/1), in particular I've reimplemented the same architecture proposed in this paper. The problem, as you can see, is ...
• 437
32 views

### Is there such a thing as intra-sample modal collapse in GANs?

Mode collapse is a known issue in generative adversarial networks (GANs) whereby the generator only learns a subset of the real data distribution. In those cases, it only outputs variations of a small ...
• 169
29 views

### How to evaluate quality of VAEs generated samples

I have a set of generated samples from a latent distribution (say 100 images) from a learned VAE. For GANs, the Inception score metric (which helps assess image quality and image diversity). Any idea ...
1 vote
39 views

### Synthetic data generation - GANs vs Simulator?

For synthetic data generation, does the GAN perform better than a simulator? If so, what are the limitations of the simulator? If we consider Conditional GANs, we could generate data based on the ...
• 11
8 views

### Conventional autoencoder training instability [duplicate]

I am currently writing an autoencoder in python (torch); its encoder is intended to serve as a compression tool. The input dataset contains a mix of numerical data (including large integers), ...
15 views

### Clarification on generative modeling

As far as I understood, discriminative models are all about finding $p(y|x)$, the conditional distribution of target labels $y$ given observations $x$ whereas generative models focus on $p(x|y)$, the ...
• 113
92 views

### Computing a prior from two components in Naive Bayes

Given a model parameter $\theta$ that is composed of two distributions in a Naive Bayes classifier, how is $P(\theta)$ typically computed in practice? More specifically, from the article of Nigam et ...
• 212
83 views

### Why generative models are better at detecting outliers?

I've read somewhere that generative models are better than discriminative ones to detect outliers in our dataset—why is that true? I think its somehow related to decision boundaries and ...
• 113
69 views

### I need help understanding the meaning of the loss values of a WGAN with Gradient Penalty

I am currently working on training a Auxiliary Classifier Wasserstein GAN with Gradient Penalty. I based my implementation off of https://keras.io/examples/generative/wgan_gp/ (to which I added the ...
1 vote
30 views

### What should I expect if I train a Variational Autoencoder (VAE) with a dataset composed of identical images?

(leaving aside how pointless this might be) Am I right in thinking that, in theory, if I train a VAE with only one image (passing it over and over), the VAE should learn to recreate that image (or a ...
• 95
1 vote
388 views

### Is k-means a generative model and how could it be used to generate new data then?

In today's lecture we learnt that k-means would be generative model. I am really puzzled on this because in my intuition it would be more a discriminative model since there is no probability to ...
14 views

### Naive Bayes for data generation

NB is a classification method which according to Bishop's book is categorized in probabilistic generative methods. As far as I understood you can learn a join distribution from input-output pairs and ...
101 views

### loss function that penalizes empirical CDF

I have been doing literature review of generative models. From what I gather, there are likelihood based generative models that model the likelihood and use it as objective function to learn the ...
• 165
1 vote
109 views

### Issues with GAN and VAE models

I'm reading this amazing paper on Normalizing Flows https://arxiv.org/pdf/1908.09257.pdf but one sentence kind of bothers me: GANs and VAEs have demonstrated impressive performance results on ...
27 views

### Do GANs compute a posterior distribution, and if not how do they have such good results with just MLE?

As the title states, do GANs (Generative Adversarial Networks) compute a posterior distribution? If they do not, how do they have such good results with just using MLE? Wouldn't they run into issues ...
• 179
1 vote
31 views

• 215
30 views

### Can a Simple ANN be Generative?

If a simple ANN was trained to predict the next step in a sequence, such as a univariate time series, can it be considered a generative model?
325 views

• 2,206