Questions tagged [generative-models]

A probabilistic or statistical model thought about as describing how the values in a sample is actually generated, and not only as a description or approximation.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

What is the learning procedure and purpose of class embeddings in Conditional GAN's

I was learning about conditional GAN's, and there is a point I did not understand about the training process. To specify the class we want to generate images from, we specify a number, which is then ...
user avatar
0 votes
0 answers
11 views

What is the exact role of model $p_\theta$ in Diffusion models for the reverse process?

I'm reading this interesting blog post explaining Diffusion probabilistic models and trying to understand the following. In order to compute the reverse process, we need to consider the posterior ...
user avatar
4 votes
2 answers
640 views

Why Logistic Regression is not a generative model?

I was reading about the difference between discriminative and generative models, and I read that Discriminative models learn only the boundary between classes hence they are not able to to create new ...
user avatar
1 vote
1 answer
35 views

How to reconstruct a Euclidean distance matrix from grouped pairwise-distance means and standard deviations?

Coordinates and Labels Take the simple case of 3 distinct object classes and 5 instances of each class situated in 3D Euclidean space. The coordinates and labels might look like the following: ...
user avatar
1 vote
0 answers
7 views

How does the fixed point interation in invertible resnets work?

I feel like I am missing some easy point about this invertible resnet paper which is making it hard for me to grasp how the fixed point iteration works. stated simply, the residual connection in a ...
user avatar
  • 629
1 vote
0 answers
61 views

Classical VAE not learning 2D gaussian mixture distribution using MSE loss

I've been exploring VAE for non-image data. I consider small to medium-sized continuous vector spaces and I want to learn the distribution of a dataset in that space. As a warm up exercise, I tried ...
user avatar
  • 113
1 vote
1 answer
23 views

Why do we use the same parameters for the joint, marginal and conditional distributions in VAEs?

I've noticed in several resources on variational autoencoders (for example the Wikipedia article), we use the same parameters theta ($\theta$) for the prior, likelihood, posterior, etc distributions. ...
user avatar
  • 11
0 votes
0 answers
21 views

Techniques Used for DeepFake and Its Corresponding Research Field

I'm a beginner in image generative models, I'm trying to do some work similar to DeepFake, therefore I would like to find out first what techniques DeepFake use to generate the fake videos. Do they ...
user avatar
  • 1,663
1 vote
0 answers
39 views

Does a CNN always learn a latent space?

In general, a latent space is a structure of reduced dimensionality than that of the input space where points on this space share resemblance the closer they are to each other. This article also ...
user avatar
  • 649
0 votes
0 answers
15 views

Using Inception and FID scores in training?

Is it possible to use the Inception and FID scores in the training of a deep image generation model, i.e. to maximize the scores in a loss function, albeit this is "cheating"? If so, has ...
user avatar
  • 1
1 vote
0 answers
33 views

GANs - why does the generator want to minimize the loss (intuition)

I am a little bit puzzled about the following. In a generative adversarial network, we consider a binary classification problem with a binary cross-entropy loss. Now, the generator wants to minimize ...
user avatar
  • 111
2 votes
1 answer
91 views

Should I be using batchnorm and/or dropout in a VAE or GAN?

I am trying to design some generative NN models on datasets of RGB images and was debating on whether I should be using dropout and/or batch norm. Here are my thoughts (I may be completely wrong): ...
user avatar
1 vote
1 answer
54 views

Initialization of GAN discriminator

The question is pretty straightforward: how are GAN and WGAN discriminators typically initialized? I couldn't find much info on this. E.x. for GANs, I imagine you would theoretically want the ...
user avatar
  • 133
2 votes
0 answers
52 views

Restricted Boltzmann Machine: W matrix visualization results after training MNIST images and Pseudo-log-likelihood

I am implementing RBM from scratch using Tensorflow and after training my RBM on the MNIST dataset for 200 epochs using Persistent CD with two steps of contrastive divergence, I learn the weights W ...
user avatar
  • 21
1 vote
0 answers
44 views

How to estimate joint probability or conditional probability using marginal ones?

I have 2 datasets: The 1st one gives us the probability that $m$ events occur on $n$ observations ($m$ columns for $n$ rows) The 2nd one tells us if the event occurred (1 if occurred, 0 else ; always ...
user avatar
0 votes
0 answers
33 views

A Generative model for binary classification (Modeling, Network architecture

I'm trying to build a network to classify input $X$ into 2 categories with a generative manner. This is because there is harsh imbalance issue in data. But I can't understand how to train the model. ...
user avatar
  • 1
0 votes
0 answers
11 views

Problem for training Wasserstein GAN

I'm trying to train a Wasserstein GAN to guess sparse one-hot encoded matrices (0/1), in particular I've reimplemented the same architecture proposed in this paper. The problem, as you can see, is ...
user avatar
0 votes
1 answer
32 views

Is there such a thing as intra-sample modal collapse in GANs?

Mode collapse is a known issue in generative adversarial networks (GANs) whereby the generator only learns a subset of the real data distribution. In those cases, it only outputs variations of a small ...
user avatar
0 votes
0 answers
29 views

How to evaluate quality of VAEs generated samples

I have a set of generated samples from a latent distribution (say 100 images) from a learned VAE. For GANs, the Inception score metric (which helps assess image quality and image diversity). Any idea ...
user avatar
  • 1
1 vote
0 answers
39 views

Synthetic data generation - GANs vs Simulator?

For synthetic data generation, does the GAN perform better than a simulator? If so, what are the limitations of the simulator? If we consider Conditional GANs, we could generate data based on the ...
user avatar
  • 11
0 votes
0 answers
8 views

Conventional autoencoder training instability [duplicate]

I am currently writing an autoencoder in python (torch); its encoder is intended to serve as a compression tool. The input dataset contains a mix of numerical data (including large integers), ...
user avatar
0 votes
0 answers
15 views

Clarification on generative modeling

As far as I understood, discriminative models are all about finding $p(y|x)$, the conditional distribution of target labels $y$ given observations $x$ whereas generative models focus on $p(x|y)$, the ...
user avatar
  • 113
0 votes
1 answer
92 views

Computing a prior from two components in Naive Bayes

Given a model parameter $\theta$ that is composed of two distributions in a Naive Bayes classifier, how is $P(\theta)$ typically computed in practice? More specifically, from the article of Nigam et ...
user avatar
0 votes
2 answers
83 views

Why generative models are better at detecting outliers?

I've read somewhere that generative models are better than discriminative ones to detect outliers in our dataset—why is that true? I think its somehow related to decision boundaries and ...
user avatar
  • 113
0 votes
0 answers
69 views

I need help understanding the meaning of the loss values of a WGAN with Gradient Penalty

I am currently working on training a Auxiliary Classifier Wasserstein GAN with Gradient Penalty. I based my implementation off of https://keras.io/examples/generative/wgan_gp/ (to which I added the ...
user avatar
1 vote
0 answers
30 views

What should I expect if I train a Variational Autoencoder (VAE) with a dataset composed of identical images?

(leaving aside how pointless this might be) Am I right in thinking that, in theory, if I train a VAE with only one image (passing it over and over), the VAE should learn to recreate that image (or a ...
user avatar
1 vote
1 answer
388 views

Is k-means a generative model and how could it be used to generate new data then?

In today's lecture we learnt that k-means would be generative model. I am really puzzled on this because in my intuition it would be more a discriminative model since there is no probability to ...
user avatar
0 votes
0 answers
14 views

Naive Bayes for data generation

NB is a classification method which according to Bishop's book is categorized in probabilistic generative methods. As far as I understood you can learn a join distribution from input-output pairs and ...
user avatar
3 votes
1 answer
101 views

loss function that penalizes empirical CDF

I have been doing literature review of generative models. From what I gather, there are likelihood based generative models that model the likelihood and use it as objective function to learn the ...
user avatar
1 vote
1 answer
109 views

Issues with GAN and VAE models

I'm reading this amazing paper on Normalizing Flows https://arxiv.org/pdf/1908.09257.pdf but one sentence kind of bothers me: GANs and VAEs have demonstrated impressive performance results on ...
user avatar
0 votes
0 answers
27 views

Do GANs compute a posterior distribution, and if not how do they have such good results with just MLE?

As the title states, do GANs (Generative Adversarial Networks) compute a posterior distribution? If they do not, how do they have such good results with just using MLE? Wouldn't they run into issues ...
user avatar
1 vote
0 answers
31 views

Neural Networks with Tractable Integral [closed]

I'm looking for neural networks $n_\theta(x)$ which integrate to a constant $\int_{[0,1]^d} n(x)\; dx=c\in\mathbb{R}$. Notably, the constant should be the same for any weights $\theta$ for which $n_\...
user avatar
0 votes
0 answers
46 views

Anomaly detection given conditional log likelihood

I have a trained generative model (normalizing flow) that gives me conditional log-likelihoods of any point $x_n$ given a set of points $y={y_0,y_1...}$. My goal is to do anomaly detection but it is ...
user avatar
1 vote
1 answer
34 views

Is the density of PixelCNN normalized?

PixelCNN++ constructs a model distribution $p(x)$ over images $x\in\mathbb{R}^{n\times n}$ as a product of conditional distributions over pixels $$p(x)=p(x_1,...,x_{n^2})=\prod_{i=1}^{n^2} p(x_i| x_1,....
user avatar
2 votes
0 answers
91 views

What is the "lower bound average gain" metric used in GMM stopping criterion used in Scikit learn?

In Scikit Learn's GMM class, it says that GMM training algorithm stops according to the "lower bound average gain" https://scikit-learn.org/stable/modules/generated/sklearn.mixture....
user avatar
  • 1,081
0 votes
0 answers
78 views

what is the number of bits assigned by a generative model to a K-class dataset, in theory

Consider a dataset, something like MNIST or CIFAR-10, that consists of (for example) 50000 images taken from 10 classes and balanced with an equal number in each class. For a discriminitive classifier,...
user avatar
1 vote
1 answer
68 views

How does fitting a generative model ($P(X,Y)$) allow one to generate examples $(X,Y)$?

For example, suppose I have a database of images of cats ($C$) and dogs ($D$). My database of labeled images consists of $(X,Y)$ where $X$ is a pixelated image and $Y \in \{C,D\}$. If I somehow fit a ...
user avatar
3 votes
0 answers
59 views

For a generative model, how is modelling p(X,Y) equivalent to modelling P(X|Y=y)?

On the Wikipedia page for generative models it gives the following definitions of a generative model: (X is an observable variable, Y is the target variable) 1) A generative model is a model of the ...
user avatar
2 votes
1 answer
523 views

Normalizing flow training

I've been learning about normalizing flow. This is my understanding, and please correct me whenever I am wrong. There are $\{y_1,y_2,...,y_n\}$ samples from an unknown distribution $p_y(y)$ that we ...
user avatar
  • 155
2 votes
2 answers
239 views

Understanding step in proof of GAN algorithm convergence, involving convexity

I am reading the original paper on GANs, https://arxiv.org/abs/1406.2661. The proof of proposition 2, on the convergence of the gradient descent algorithm reads Consider $V(G, D) = U(p_g, D)$ as a ...
user avatar
2 votes
2 answers
166 views

What concept comes before VAE and after GMM?

Suppose I am designing a course on generative models and I have just finished discussing GMM. My goal is to teach VAE. However, VAE's technicality is very high. Does there exist some model in between ...
user avatar
  • 674
1 vote
0 answers
43 views

Normalizing Flows notations and change of variable formula

Maybe it is a silly question, but I am wondering why in some papers the function $f$ is the mapping from $x$ to $z$ and in some others it is the other way around. Typically in this review we go from $...
user avatar
  • 215
2 votes
1 answer
30 views

Can a Simple ANN be Generative?

If a simple ANN was trained to predict the next step in a sequence, such as a univariate time series, can it be considered a generative model?
user avatar
2 votes
1 answer
325 views

Are text generation models generative or discriminative?

I've recently been studying generative and discriminative models, and I had a question regarding text generation. I'm aware that generative models model $P(X, Y)$ and discriminative models model $P(Y |...
user avatar
  • 2,471
3 votes
1 answer
437 views

How to find bits/dim of a gaussian output distribution?

I have images that are 64x64x3 and 64x64x1 8-bit. I transform those images down to [-1,1]. I now want to find the bits/dim for my VAE log probability. How do I find the bits/dim of the log likelihood? ...
user avatar
  • 65
2 votes
1 answer
11 views

Updating of characer's embedding inside an RNN during character generation task

In one of the tutorials of tensorflow, there is a "text generation with an RNN" tutorial. When creating the model, they create a mapping of characters to IDs and vice versa. Then in the ...
user avatar
  • 385
1 vote
1 answer
763 views

When is my Wasserstein GAN-GP overfitting?

I have a hard time interpreting the WGAN-GP losses attached. At which epoch is D and/or G overfitting? The quality improves a lot overtime, yet the generator loss at later epochs does not appear on ...
user avatar
1 vote
1 answer
74 views

Latent Dirichlet Allocation - dimensionality of the Dirichlet prior parameter

I seeking some clarity on the dimensionality of the (hyper)parameter $\eta$ of the "smoothed LDA" model in Section 5.4 of the original paper by Blei, Ng, Jordan (2003), which can be found ...
user avatar
  • 2,206
-1 votes
1 answer
53 views

K-Lipschitz for the Wasserstein GANs

I am trying to follow this blog for Wasserstein loss for Generative Adversarial Networks: From GAN to WGAN. Actually, I am trying to follow the logic behind the K-Lipschitz continuity. This post in ...
user avatar
2 votes
1 answer
3k views

Why use tanh function at the last layer of generator in GAN?

While studying GAN, I found out that ReLU activation is used at the intermediate layers, and tanh or sigmoid is used at the last layer of the generator. I'm curious about why sigmoid or tanh is used ...
user avatar
  • 199

1
2 3 4 5