Questions tagged [generative-models]

A probabilistic or statistical model thought about as describing how the values in a sample is actually generated, and not only as a description or approximation.

Filter by
Sorted by
Tagged with
2
votes
1answer
1k views

Generating distributions with with given variance, skewness, and kurtosis

I would like to generate distributions that are as close to normal as possible, except for the deviations shows below. The options I've located have properties that are related to the families of ...
0
votes
1answer
38 views

Generating text using LSTM given condition vector

I know that you can use an RNN to generate text given the first few letters ...
2
votes
0answers
20 views

Softmax classifier with class priors

Softmax classifier is a discriminative model that directly models $p(Y|X, w)$ where $Y$ is the label for input $X$. We can write it as follows: $$p(y_i=k|x_i,w_k)=\frac{\exp \lbrace w_k^T x_i \rbrace}{...
2
votes
0answers
32 views

Extremely basic question: how are data assumed to be generated in machine learning?

Given a dataset $\mathcal{D} = \{x_i\}, i = 1, \ldots, N, x_i \in \mathbb{R}$ In machine learning, what assumption is made as to how data are generated? I've seen two basic ideas circulating around, ...
1
vote
1answer
20 views

How do we generate the samples of hidden root nodes in the Bayes network (Sigmoid Belief Networks) of a generative model

Following is a Sigmoid Belief Networks where we can only observe the bottom observable layer $v.$ Usually we use ...
0
votes
1answer
119 views

Distribution of Data in GAN

Will Generator and Discriminator learns the Distribution of Data when it was trained? If So Both Generator and Discriminator learns same distribution or different distributions ? What can we interpret ...
5
votes
1answer
3k views

How to interprete Discriminator and Generator loss in GAN

I trained GAN with learning rate 0.00002, discriminator is trained once and generator is trained twice per epoch. Wasserstein loss is used as loss function This is the loss graph for discriminator ...
1
vote
0answers
27 views

What are the best known techniques to verify that a GAN samples correctly from a given distribution?

I would like to know what are the best known techniques to check that a generative adversarial network (GAN) samples from the correct distribution. Naively, I would say it all boils down to a ...
0
votes
1answer
495 views

Understanding reparameterization trick and training process in variational autoencoders

I am trying to understand variational autoencoders, particularly the sampling component and the reparameterization trick. I understand that instead of using a fixed determinstic latent representation ...
1
vote
0answers
324 views

Uniformly distributed VAE samples

I am currently working on a VAE to generate images (for simplicity MNIST). If I understand the theory correctly, the latent variables follow a gaussian normal distribution in the dimensions of the ...
1
vote
0answers
34 views

What is the intution behind generative and discriminative models?

Lets say we have a dataset $D=(X,Y)$ and our aim is to find a model $f$ which maps our feature and target. $f:X \rightarrow Y$ According to wikipedia, A discriminative model is a model of the ...
1
vote
0answers
153 views

Optimal critic in Wasserstein GAN (WGAN)

In the Wasserstein GAN (generative adversarial network) paper they say that the critic needs to be optimal. What does this mean practically? They do more training iterations of critic (5) than the ...
1
vote
1answer
266 views

Can someone explain CAM loss used in U-GAT-IT paper?

I have been reading a recent paper accepted at ICLR, U-GAT-IT, which seems to produce pleasing results in the image-to-image translation tasks. There are four kinds of loss used in this paper: ...
5
votes
1answer
359 views

How do you calculate log likelihood p(x) for a VAE?

I was reading the Importance Weighted Autoencoders paper and its experiment section compares the density estimation result on MNIST for IWAE vs VAE. I know that density estimation estimating log p(x) ...
1
vote
1answer
94 views

Differentiability in Generative Adversarial Networks

I've got some questions about the differentiability condition of GAN's, i.e. both G and D need to be differentiable wrt. their inputs and the parameters describing them. It's of more mathematical ...
1
vote
1answer
24 views

regress on to a design matrix sampled from a distribution

Just wondering if I am thinking of this correctly. I would really appreciate any comments on the approach. I have a problem where I need to regress the outcome ($y$) on a design matrix ($X$). So, $$...
2
votes
1answer
431 views

What is the advantages of minimizing Wasserstein divergence rather than Pearson divergence in a GAN?

Generative Adversarial Networks (GANs) are generative models that jointly train two neural networks: a discriminator, that learns to say apart real data from generated data, and a generator, that ...
0
votes
1answer
94 views

why predict a distribution in pixelcnn++ [closed]

I know that in the original pixelcnn paper, they predicted a 255 vector for each subpixel, and argmaxed to get the value. in the pixelcnn++ paper, if I understand it correctly, they model the pixel ...
2
votes
1answer
276 views

Perceptual Loss Layers Selection

I understand that in order to improve your generative model performance it is quite useful to compare your output and the target in the feature space, as stated in the paper Perceptual Losses for Real-...
0
votes
0answers
298 views

Kernel density estimates comparison in a multivariate setting

I have a dataset with 64 features and binary labels (class 1 and class 2). Before I fit any classification models, I wanted to check whether the samples belonging to the two classes come from the same ...
1
vote
1answer
43 views

why is using a mixture of logstic distributions makes sense in pixelcnn++?

I went trough the paper and code of the pixelcnn++ model. From what I understand, they train the network in the following way for predicting the value of a single pixel: the inputs are the pixel ...
3
votes
2answers
1k views

Planar Flow in Normalizing Flows

While I've read "Variational Inference with Normalizing Flows" (abstract), I don't understand about an intuition of Planar Flow. The author defined Planar Flow as below Let $\boldsymbol{w} \in \...
1
vote
1answer
766 views

A meaning/significance of validation loss in a Generative Adversarial Neural Network? [closed]

On most of the tutorials on GANs that I came across the only monitored quantity is training loss. 1) Are there any general conclusions that could be derived from comparing training and validation ...
1
vote
0answers
45 views

About Feature Engineering Tips When “discriminative algorithm care about no modeling the probability of the language”

I was going over my old NLP course slides and one of the pages is about using Structured Perceptron for tagging. It claims that because the algorithm is discriminative, it doesn't care about modeling ...
1
vote
1answer
192 views

How do I resize output of a generative network to reconcile with the size of the real data?

I am designing a Generative Adversarial Network (GAN) trained on an image dataset. It has two components: the generator and the discriminator. The generative network outputs an artificial image. The ...
2
votes
2answers
533 views

What's the advantage of multi-scale architecture in Normalizing Flow?

While I've read a paper "Density Estimation using Real NVP", I have some confusing parts about multi-scale architecture in Section 3.6. 1. The author said that We implement a multi-scale ...
0
votes
1answer
34 views

Difference / Relationship of Generative Models / Variational Bayesian Inference

I feel a bit confused trying to merge and unify understandings of generative models and variational bayesian inference methods. Initially, I believed them to be the same thing, namely learning full ...
0
votes
0answers
33 views

GAN loss doesnt ocsillate but converges relly quickly

I am trying to train a GAN and am getting a strange convergence issue. The losses seem to converge really quickly and dont ocsilate. The examples of the generator produces are noise. Both the ...
1
vote
0answers
35 views

One shot inference with Variational Autoencoders using proposal mean

Let's say you have an already trained Variational Autoencoder where the parameters are $\phi, \theta$ for the recognition and generative models respectively. Let's also assume you have the following ...
3
votes
1answer
40 views

Why are autoregressive models neither directed or undirected, as described in the NADE paper?

In the paper Neural Autoregressive Distribution Estimation (Uria et al., 2016), NADE (and other autoregressive models) seem to be described as neither directed or undirected models: We’ve described ...
3
votes
0answers
169 views

Are there nonparametric generative models for datasets?

Typically when I see generative models, e.g., Latent Dirichlet Allocation (JMLR) or Linear/Quadratic Discriminant Analysis (wikipedia LDA), they are probabilistic models that belong to the exponential ...
4
votes
2answers
65 views

Is there a measure or notion of correlation (or association) without assuming finite second order moments?

When we study correlation or association in real data, do we always (implicitly) assume a finite second order moment for any hypothetical population distribution? If we do not assume this, what ...
2
votes
1answer
227 views

Controlling details in images generated by Generative Adversarial Networks (GANs)

With conditional GANs it is possible to generate images of a certain class of objects. And moreover with current text-to-image methods it seems to be possible to control certain details of the ...
0
votes
0answers
102 views

Generating Graphs using a Neural Network

I currently have constructed a Graph Neural Network in PyTorch with graph conv layers I have made. With this, I am able to feed in adjacency and feature matrices and successfully perform node ...
1
vote
1answer
60 views

Which is the decision rule in a gda classifier?

On the text book there is the following formula for the prediction rule, but I don’t understand where it comes from: The textbooks says it had been derived from I suppose pi and theta are the ...
3
votes
1answer
344 views

Generative design - algorithms and sources

I have recently encountered the term "generative design" where a computer algorithm (usually with the help of Machine Learning) comes up with new designs that conform to a certain set of requirements. ...
11
votes
2answers
3k views

What are the current methods to check for GAN overfitting?

In generative modeling, the goal is to find a way for a model to output samples of some distribution $p_X$ given a lot of samples $x_1, \ldots, x_n$. In particular, we want sampling from our model $G$ ...
1
vote
1answer
35 views

Basic doubt on generative models

I am new to statistics and while reading Bishop's book, in the 'Generative models' part 8.1.2. When explaining ancestral sample, he says: To do this, we start with the lowest-numbered node and ...
4
votes
3answers
369 views

What methods can be used for distribution generation other than GANs?

Generative Adversarial Networks (GANs) can be used for creating distributions of data points, that follow source data set distributions (e.g. images, sound, text, etc). Are there any other methods or ...
1
vote
0answers
94 views

Does discriminator converge to probability 0.5 in GAN just for the output of generator?

We know that in GANs, discriminator converges to 0.5. I want to become sure that the following statement is true: Discriminator converges to 0.5 for the examples generated by generator and for the ...
2
votes
2answers
35 views

How to extract crucial features to create an image

Imagine, you have a dataset containing pictures of (example only, just to explain the task) cats and dogs. The data set is labeled, so we can train using supervised learning algorithms. My goal is ...
8
votes
1answer
124 views

Structure of Generative Adversarial Networks (GAN) for mapping a simulation model

There is a simulation model of a system that I want to map as a neural network to test if a better execution time can be achieved with similar accuracy. The simulation model receives real-valued ...
2
votes
1answer
226 views

Reparametrization trick in VAE at Evaluation time

So I've been trying to implement the Variational Auto-Encoder model of Kigma et.al, but something has been bugging me. While I understand the need for reparametrization trick at training time, the ...
4
votes
1answer
107 views

Deep generative models learning a Bayesian-network distribution

Say I have a generative model for some distribution $p$ over a small number of RVs which allows me to easily sample from said distribution. For example, let's say it's a parameterized Bayesian network ...
2
votes
1answer
76 views

How to test generated text

I am creating a text generation algorithm for my master's research. I have a dialogue between two people and I would like to simulate one part of the conversation with naturally generated text (not ...
12
votes
1answer
557 views

How to understand Generative Adversarial Networks Discriminative distribution?

So I am currently studying Generative Adversarial Network and I read the paper by Goodfellow a few times now Generative Adversarial Nets and a few other papers in this field (DCGAN, CycleGAN, pix2pix, ...
1
vote
0answers
132 views

Are Flow Based Generative Models Referring to the Invertible Transformations?

So I have been studying generative models for a while now. I know how GANs and VAEs work quite well, but I am quite confused by how Flow Based Generative Models work. To my understanding, flow based ...
1
vote
0answers
30 views

Why generative models in Machine Learning are Boltzmann distribution-backed?

I learned from this review paper that MaxEnt models naturally display a Boltzmann distribution for the data samples, it comes from the Principle of Maximum Entropy. But I could not understand why this ...
0
votes
1answer
124 views

Weak Supervision - training generative model without knowing the true label

Recently I've been reading about weak supervision. I understand most of the concept details, there's one thing that is not clear to me though. In the generative model part (creating generative model ...
1
vote
1answer
72 views

How to generate random samples from a 2D dataset?

Suppose I'm given a data set consisting of many pairs of $(x,y)$ values which are correlated in some arbitrary complex way. How would I go about 'generating' more pairs of $(x,y)$ coordinates which ...