Questions tagged [generative-models]

A probabilistic or statistical model thought about as describing how the values in a sample is actually generated, and not only as a description or approximation.

Filter by
Sorted by
Tagged with
1
vote
0answers
8 views

LSTM high accuracy but poor generation performance

I'm writing a LSTM model for generating music (in particular drums). My model is based on these 2 models: LSTM text generator LSTM drum generator The model seems to work fine, it trains and I can ...
0
votes
0answers
6 views

Can Gaussian mixture models help an algorithm target a specific cluster?

In the chart below is a Gaussian Mixture model (GMM) based on three time series or datasets that the model was able to easily cluster into three different colored classes. Class 1 is the blue ellipse ...
0
votes
0answers
19 views

What are the best known techniques to verify that a GAN samples correctly from a given distribution?

I would like to know what are the best known techniques to check that a generative adversarial network (GAN) samples from the correct distribution. Naively, I would say it all boils down to a ...
0
votes
1answer
13 views

Understanding reparameterization trick and training process in variational autoencoders

I am trying to understand variational autoencoders, particularly the sampling component and the reparameterization trick. I understand that instead of using a fixed determinstic latent representation ...
0
votes
0answers
36 views

Uniformly distributed VAE samples

I am currently working on a VAE to generate images (for simplicity MNIST). If I understand the theory correctly, the latent variables follow a gaussian normal distribution in the dimensions of the ...
1
vote
0answers
23 views

What is the intution behind generative and discriminative models?

Lets say we have a dataset $D=(X,Y)$ and our aim is to find a model $f$ which maps our feature and target. $f:X \rightarrow Y$ According to wikipedia, A discriminative model is a model of the ...
0
votes
1answer
37 views

Can someone explain CAM loss used in U-GAT-IT paper?

I have been reading a recent paper accepted at ICLR, U-GAT-IT, which seems to produce pleasing results in the image-to-image translation tasks. There are four kinds of loss used in this paper: ...
2
votes
0answers
24 views

How do you calculate log likelihood p(x) for a VAE?

I was reading the Importance Weighted Autoencoders paper and its experiment section compares the density estimation result on MNIST for IWAE vs VAE. I know that density estimation estimating log p(x) ...
1
vote
1answer
14 views

Differentiability in Generative Adversarial Networks

I've got some questions about the differentiability condition of GAN's, i.e. both G and D need to be differentiable wrt. their inputs and the parameters describing them. It's of more mathematical ...
1
vote
1answer
16 views

regress on to a design matrix sampled from a distribution

Just wondering if I am thinking of this correctly. I would really appreciate any comments on the approach. I have a problem where I need to regress the outcome ($y$) on a design matrix ($X$). So, $$...
2
votes
1answer
37 views

What is the advantages of minimizing Wasserstein divergence rather than Pearson divergence in a GAN?

Generative Adversarial Networks (GANs) are generative models that jointly train two neural networks: a discriminator, that learns to say apart real data from generated data, and a generator, that ...
0
votes
1answer
17 views

why predict a distribution in pixelcnn++ [closed]

I know that in the original pixelcnn paper, they predicted a 255 vector for each subpixel, and argmaxed to get the value. in the pixelcnn++ paper, if I understand it correctly, they model the pixel ...
0
votes
1answer
24 views

Perceptual Loss Layers Selection

I understand that in order to improve your generative model performance it is quite useful to compare your output and the target in the feature space, as stated in the paper Perceptual Losses for Real-...
0
votes
0answers
21 views

Kernel density estimates comparison in a multivariate setting

I have a dataset with 64 features and binary labels (class 1 and class 2). Before I fit any classification models, I wanted to check whether the samples belonging to the two classes come from the same ...
1
vote
1answer
17 views

why is using a mixture of logstic distributions makes sense in pixelcnn++?

I went trough the paper and code of the pixelcnn++ model. From what I understand, they train the network in the following way for predicting the value of a single pixel: the inputs are the pixel ...
3
votes
2answers
105 views

Planar Flow in Normalizing Flows

While I've read "Variational Inference with Normalizing Flows" (abstract), I don't understand about an intuition of Planar Flow. The author defined Planar Flow as below Let $\boldsymbol{w} \in \...
1
vote
1answer
37 views

A meaning/significance of validation loss in a Generative Adversarial Neural Network? [closed]

On most of the tutorials on GANs that I came across the only monitored quantity is training loss. 1) Are there any general conclusions that could be derived from comparing training and validation ...
1
vote
0answers
38 views

About Feature Engineering Tips When “discriminative algorithm care about no modeling the probability of the language”

I was going over my old NLP course slides and one of the pages is about using Structured Perceptron for tagging. It claims that because the algorithm is discriminative, it doesn't care about modeling ...
0
votes
1answer
17 views

How do I resize output of a generative network to reconcile with the size of the real data?

I am designing a Generative Adversarial Network (GAN) trained on an image dataset. It has two components: the generator and the discriminator. The generative network outputs an artificial image. The ...
1
vote
2answers
123 views

What's the advantage of multi-scale architecture in Normalizing Flow?

While I've read a paper "Density Estimation using Real NVP", I have some confusing parts about multi-scale architecture in Section 3.6. 1. The author said that We implement a multi-scale ...
0
votes
1answer
27 views

Difference / Relationship of Generative Models / Variational Bayesian Inference

I feel a bit confused trying to merge and unify understandings of generative models and variational bayesian inference methods. Initially, I believed them to be the same thing, namely learning full ...
0
votes
0answers
17 views

Small handwriting database generation and augmentation

I am willing to extend my handwriting text recognition (HTR) offline system with some Spanish characters. So far, I have trained with images from the IAM Database, which includes english characters. ...
0
votes
0answers
19 views

Machine learning a discrete probability distribution that is parametrized by a set of real-valued parameters

Assume I have a probability distribution $p_\theta(\sigma)$ defined over discrete binary variables $\sigma$, $p_\theta(\sigma) : \sigma \to [0,1]$. This probability distribution is parametrized by a ...
0
votes
0answers
7 views

Generating Fixed Length Sequence with RNNs

Is there any way of generating fixed-length sequences with RNNs? I want to tell my character level RNN to generate a name of length 3, 4, 5 and so on. I haven't found anything online like this, but my ...
0
votes
0answers
23 views

GAN loss doesnt ocsillate but converges relly quickly

I am trying to train a GAN and am getting a strange convergence issue. The losses seem to converge really quickly and dont ocsilate. The examples of the generator produces are noise. Both the ...
1
vote
0answers
27 views

One shot inference with Variational Autoencoders using proposal mean

Let's say you have an already trained Variational Autoencoder where the parameters are $\phi, \theta$ for the recognition and generative models respectively. Let's also assume you have the following ...
0
votes
0answers
13 views

Generative capability of autoencoders and distribution modelling

I've been reading up quite a lot on autoencoders and variational inference. VAEs are used to generate data in accordance with the distribution of the training data. I am unable to understand how ...
2
votes
1answer
28 views

Why are autoregressive models neither directed or undirected, as described in the NADE paper?

In the paper Neural Autoregressive Distribution Estimation (Uria et al., 2016), NADE (and other autoregressive models) seem to be described as neither directed or undirected models: We’ve described ...
2
votes
0answers
40 views

Are there nonparametric generative models for datasets?

Typically when I see generative models, e.g., Latent Dirichlet Allocation (JMLR) or Linear/Quadratic Discriminant Analysis (wikipedia LDA), they are probabilistic models that belong to the exponential ...
4
votes
2answers
45 views

Is there a measure or notion of correlation (or association) without assuming finite second order moments?

When we study correlation or association in real data, do we always (implicitly) assume a finite second order moment for any hypothetical population distribution? If we do not assume this, what ...
1
vote
1answer
131 views

Controlling details in images generated by Generative Adversarial Networks (GANs)

With conditional GANs it is possible to generate images of a certain class of objects. And moreover with current text-to-image methods it seems to be possible to control certain details of the ...
0
votes
0answers
50 views

Generating Graphs using a Neural Network

I currently have constructed a Graph Neural Network in PyTorch with graph conv layers I have made. With this, I am able to feed in adjacency and feature matrices and successfully perform node ...
0
votes
0answers
33 views

Implementing the GAN loss function

I was reading the original Generative Adversarial Nets paper by Goodfellow et al. This is the minimax optimization problem for GANs: All the code I have seen, however, seem to have a different ...
0
votes
0answers
13 views

Why are Auxiliary Classifier GANs useful?

I am new to GANs and am learning about the different types of GANs. What added value do ACGANs provide over standard GANs and when should I use them? What is the difference conceptually between using ...
1
vote
1answer
25 views

Which is the decision rule in a gda classifier?

On the text book there is the following formula for the prediction rule, but I don’t understand where it comes from: The textbooks says it had been derived from I suppose pi and theta are the ...
3
votes
1answer
132 views

Generative design - algorithms and sources

I have recently encountered the term "generative design" where a computer algorithm (usually with the help of Machine Learning) comes up with new designs that conform to a certain set of requirements. ...
6
votes
2answers
323 views

What are the current methods to check for GAN overfitting?

In generative modeling, the goal is to find a way for a model to output samples of some distribution $p_X$ given a lot of samples $x_1, \ldots, x_n$. In particular, we want sampling from our model $G$ ...
0
votes
1answer
31 views

Basic doubt on generative models

I am new to statistics and while reading Bishop's book, in the 'Generative models' part 8.1.2. When explaining ancestral sample, he says: To do this, we start with the lowest-numbered node and ...
4
votes
3answers
337 views

What methods can be used for distribution generation other than GANs?

Generative Adversarial Networks (GANs) can be used for creating distributions of data points, that follow source data set distributions (e.g. images, sound, text, etc). Are there any other methods or ...
0
votes
0answers
28 views

Does discriminator converge to probability 0.5 in GAN just for the output of generator?

We know that in GANs, discriminator converges to 0.5. I want to become sure that the following statement is true: Discriminator converges to 0.5 for the examples generated by generator and for the ...
2
votes
2answers
27 views

How to extract crucial features to create an image

Imagine, you have a dataset containing pictures of (example only, just to explain the task) cats and dogs. The data set is labeled, so we can train using supervised learning algorithms. My goal is ...
8
votes
1answer
111 views

Structure of Generative Adversarial Networks (GAN) for mapping a simulation model

There is a simulation model of a system that I want to map as a neural network to test if a better execution time can be achieved with similar accuracy. The simulation model receives real-valued ...
0
votes
1answer
61 views

Reparametrization trick in VAE at Evaluation time

So I've been trying to implement the Variational Auto-Encoder model of Kigma et.al, but something has been bugging me. While I understand the need for reparametrization trick at training time, the ...
3
votes
1answer
56 views

Deep generative models learning a Bayesian-network distribution

Say I have a generative model for some distribution $p$ over a small number of RVs which allows me to easily sample from said distribution. For example, let's say it's a parameterized Bayesian network ...
2
votes
1answer
34 views

How to test generated text

I am creating a text generation algorithm for my master's research. I have a dialogue between two people and I would like to simulate one part of the conversation with naturally generated text (not ...
0
votes
0answers
18 views

How to draw CycleGANs discriminative distribution?

A few weeks ago I asked this question which was about how to understand GANs discriminative distribution. Now I am trying to draw a Figure like the one below, but for CycleGAN instead. In CycleGAN ...
0
votes
0answers
9 views

Generative Adversarial Networks evaluation methods for one channel

So I'm currently studying GANs with a focus on CycleGAN. I have trained my network on simulated images and real images. I did not train them as pairs but I have pairs available. The idea is now to ...
10
votes
1answer
509 views

How to understand Generative Adversarial Networks Discriminative distribution?

So I am currently studying Generative Adversarial Network and I read the paper by Goodfellow a few times now Generative Adversarial Nets and a few other papers in this field (DCGAN, CycleGAN, pix2pix, ...
1
vote
0answers
77 views

Are Flow Based Generative Models Referring to the Invertible Transformations?

So I have been studying generative models for a while now. I know how GANs and VAEs work quite well, but I am quite confused by how Flow Based Generative Models work. To my understanding, flow based ...
1
vote
0answers
23 views

Why generative models in Machine Learning are Boltzmann distribution-backed?

I learned from this review paper that MaxEnt models naturally display a Boltzmann distribution for the data samples, it comes from the Principle of Maximum Entropy. But I could not understand why this ...