Questions tagged [generative-models]

A probabilistic or statistical model thought about as describing how the values in a sample is actually generated, and not only as a description or approximation.

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

Inference in Normalizing Flow model: NICE(non linear independent components estimation)

I was recently reading Y. Bengio's paper on NICE (https://arxiv.org/abs/1410.8516). In the paper, authors have taken a view that a good representation involves easy learning of the data distribution. ...
0
votes
0answers
19 views

Converting expectation to probability for generative model

I'm dealing with a problem in which I have many individual samples, and each of them has a non-negative numeric outcome. I want to predict, for groups of these samples, the expected value of the sum ...
2
votes
0answers
46 views

Can PCA generate a new random image?

I read The Batch: GANs newsletter and Goodfellow said: My colleague Bing Xu modeled face images from the Toronto Face Database, which were only 90 pixels square and grayscale. Because the faces were ...
1
vote
0answers
25 views

Probabilistic Difference between Autoencoders and Variational Autoencoders

I have recently read up about Autoencoders and Variational Autoencoders. In Variational Autoencoders, the loss is modeled based on what distribution we choose for P(x|z). So, if we choose it to be ...
0
votes
0answers
10 views

Probabilistic generative models for clustering and classification

I have a question regarding the probabilistic setting of clustering and classification. More specifically regarding Gaussian Mixture Models and probabilistic generative models for classification. In ...
0
votes
1answer
36 views

Why VAE need reparameterization trick while LDA does not (both using variational inference for optimization)

Both VAE and LDA (latent Dirichlet allocation) is based on variational inference, and they both try to optimize ELBO objective function Variational autoencoders use reparameterization so that "...
0
votes
0answers
25 views

Variational Autoencoder with a Flexible Prior

Let's say I have a Variational autoencoder with a Gaussian prior architecture and I want to regularize this VAE with a flexible prior. Does sampling with the normal "reparameterization trick"...
4
votes
1answer
30 views

How well does GAN (generative adversarial network) perform for small samples?

GAN is an unsupervised learning algorithm that pits a discriminator and generator against one another so that they iteratively compete to enhance the overall model's ability to model/replicate a given ...
0
votes
1answer
48 views

In VAE, why use MSE loss between input x and decoded sample x' from latent distribution?

Variational Autoencoders (VAEs) are based on the concept of Variational Inference (VI) and use two Neural Networks similar to Vanilla Autoencoders (AEs) for function approximation. I understood the ...
1
vote
1answer
42 views

Why we learn $\log{\sigma^2}$ in VAE reparameterization trick instead of standard deviation?

We know that the reparameterization trick is to learn two vectors $\sigma$ and $\mu$, sample $\epsilon$ from $N(0, 1)$ and then your latent vector $Z$ would be (where $\odot$ is the element-wise ...
5
votes
1answer
125 views

reparameterization trick in VAEs, How should we do this?

I'm confused about how does reparameterization trick works. In this article shows it very simple. You learn two vectors $\sigma$ and $\mu$, sample $\epsilon$ from $N(0, 1)$ and then your latent ...
1
vote
1answer
39 views

Variational Autoencoder (VAE) latent features

I'm new to DL and I'm working on VAE for biomedical images. I need to extract relevant features from ct scan. So I created first an autoencoder and after a VAE. My doubt is that I don't know from ...
0
votes
1answer
24 views

What is the Intuition behind the GAN Discriminator loss? How does Discriminator loss works?

I have just stated learning GAN and the loss used are different for same problems in same tutorial. Could someone please tell me intutively that which loss function is doing what? For example, in the ...
1
vote
0answers
29 views

Generative Models and Continuous Variables

When reading about generative models such as variational autoencoders and generative adversarial networks, I see many references to the probability of a given sample, i.e. $p(x)$, where the sample of $...
0
votes
2answers
82 views

What is the stop criteria of generative adversarial nets?

I have used the GANs (Generative Adversarial Networks) with a binary cross-entropy loss function (in both generator and discriminator). Throughout the training step, the variation of generator loss ...
0
votes
0answers
23 views

Conditional generative models for large numbers of classes

I'm looking for a conditional generative model — could be cGAN or cVAE — that is particularly well suited to large numbers of classes. I'm expecting to require 500+ classes, but so many models are ...
2
votes
1answer
85 views

Generating distributions with with given variance, skewness, and kurtosis

I would like to generate distributions that are as close to normal as possible, except for the deviations shows below. The options I've located have properties that are related to the families of ...
0
votes
1answer
16 views

Generating text using LSTM given condition vector

I know that you can use an RNN to generate text given the first few letters ...
0
votes
0answers
5 views

the IS(Inception Score) of the generated images is higher than the images from datasets?

Is there a situation where the IS(Inception Score) of the generated images is higher than the images from datasets? Can anyone tell me?
2
votes
0answers
15 views

Softmax classifier with class priors

Softmax classifier is a discriminative model that directly models $p(Y|X, w)$ where $Y$ is the label for input $X$. We can write it as follows: $$p(y_i=k|x_i,w_k)=\frac{\exp \lbrace w_k^T x_i \rbrace}{...
2
votes
0answers
30 views

Extremely basic question: how are data assumed to be generated in machine learning?

Given a dataset $\mathcal{D} = \{x_i\}, i = 1, \ldots, N, x_i \in \mathbb{R}$ In machine learning, what assumption is made as to how data are generated? I've seen two basic ideas circulating around, ...
1
vote
1answer
15 views

How do we generate the samples of hidden root nodes in the Bayes network (Sigmoid Belief Networks) of a generative model

Following is a Sigmoid Belief Networks where we can only observe the bottom observable layer $v.$ Usually we use ...
0
votes
1answer
39 views

Distribution of Data in GAN

Will Generator and Discriminator learns the Distribution of Data when it was trained? If So Both Generator and Discriminator learns same distribution or different distributions ? What can we interpret ...
0
votes
0answers
14 views

proof of For $G$ fixed, the optimal discriminator $D$

This question is related to a part of Why Discriminator converges to 1/2 in Generative Adversarial Networks? I'm looking at the proof of For $G$ fixed, the optimal discriminator $D$ is $ D_{G}^{*}(\...
1
vote
0answers
13 views

LSTM high accuracy but poor generation performance

I'm writing a LSTM model for generating music (in particular drums). My model is based on these 2 models: LSTM text generator LSTM drum generator The model seems to work fine, it trains and I can ...
0
votes
0answers
11 views

Can Gaussian mixture models help an algorithm target a specific cluster?

In the chart below is a Gaussian Mixture model (GMM) based on three time series or datasets that the model was able to easily cluster into three different colored classes. Class 1 is the blue ellipse ...
2
votes
1answer
53 views

How to interprete Discriminator and Generator loss in GAN

I trained GAN with learning rate 0.00002, discriminator is trained once and generator is trained twice per epoch. Wasserstein loss is used as loss function This is the loss graph for discriminator ...
1
vote
0answers
26 views

What are the best known techniques to verify that a GAN samples correctly from a given distribution?

I would like to know what are the best known techniques to check that a generative adversarial network (GAN) samples from the correct distribution. Naively, I would say it all boils down to a ...
0
votes
1answer
36 views

Understanding reparameterization trick and training process in variational autoencoders

I am trying to understand variational autoencoders, particularly the sampling component and the reparameterization trick. I understand that instead of using a fixed determinstic latent representation ...
1
vote
0answers
86 views

Uniformly distributed VAE samples

I am currently working on a VAE to generate images (for simplicity MNIST). If I understand the theory correctly, the latent variables follow a gaussian normal distribution in the dimensions of the ...
1
vote
0answers
26 views

What is the intution behind generative and discriminative models?

Lets say we have a dataset $D=(X,Y)$ and our aim is to find a model $f$ which maps our feature and target. $f:X \rightarrow Y$ According to wikipedia, A discriminative model is a model of the ...
0
votes
0answers
29 views

Optimal critic in Wasserstein GAN (WGAN)

In the Wasserstein GAN (generative adversarial network) paper they say that the critic needs to be optimal. What does this mean practically? They do more training iterations of critic (5) than the ...
0
votes
1answer
80 views

Can someone explain CAM loss used in U-GAT-IT paper?

I have been reading a recent paper accepted at ICLR, U-GAT-IT, which seems to produce pleasing results in the image-to-image translation tasks. There are four kinds of loss used in this paper: ...
3
votes
0answers
56 views

How do you calculate log likelihood p(x) for a VAE?

I was reading the Importance Weighted Autoencoders paper and its experiment section compares the density estimation result on MNIST for IWAE vs VAE. I know that density estimation estimating log p(x) ...
1
vote
1answer
16 views

Differentiability in Generative Adversarial Networks

I've got some questions about the differentiability condition of GAN's, i.e. both G and D need to be differentiable wrt. their inputs and the parameters describing them. It's of more mathematical ...
1
vote
1answer
18 views

regress on to a design matrix sampled from a distribution

Just wondering if I am thinking of this correctly. I would really appreciate any comments on the approach. I have a problem where I need to regress the outcome ($y$) on a design matrix ($X$). So, $$...
2
votes
1answer
54 views

What is the advantages of minimizing Wasserstein divergence rather than Pearson divergence in a GAN?

Generative Adversarial Networks (GANs) are generative models that jointly train two neural networks: a discriminator, that learns to say apart real data from generated data, and a generator, that ...
0
votes
1answer
32 views

why predict a distribution in pixelcnn++ [closed]

I know that in the original pixelcnn paper, they predicted a 255 vector for each subpixel, and argmaxed to get the value. in the pixelcnn++ paper, if I understand it correctly, they model the pixel ...
0
votes
1answer
33 views

Perceptual Loss Layers Selection

I understand that in order to improve your generative model performance it is quite useful to compare your output and the target in the feature space, as stated in the paper Perceptual Losses for Real-...
0
votes
0answers
30 views

Kernel density estimates comparison in a multivariate setting

I have a dataset with 64 features and binary labels (class 1 and class 2). Before I fit any classification models, I wanted to check whether the samples belonging to the two classes come from the same ...
1
vote
1answer
19 views

why is using a mixture of logstic distributions makes sense in pixelcnn++?

I went trough the paper and code of the pixelcnn++ model. From what I understand, they train the network in the following way for predicting the value of a single pixel: the inputs are the pixel ...
3
votes
2answers
238 views

Planar Flow in Normalizing Flows

While I've read "Variational Inference with Normalizing Flows" (abstract), I don't understand about an intuition of Planar Flow. The author defined Planar Flow as below Let $\boldsymbol{w} \in \...
1
vote
1answer
87 views

A meaning/significance of validation loss in a Generative Adversarial Neural Network? [closed]

On most of the tutorials on GANs that I came across the only monitored quantity is training loss. 1) Are there any general conclusions that could be derived from comparing training and validation ...
1
vote
0answers
39 views

About Feature Engineering Tips When “discriminative algorithm care about no modeling the probability of the language”

I was going over my old NLP course slides and one of the pages is about using Structured Perceptron for tagging. It claims that because the algorithm is discriminative, it doesn't care about modeling ...
0
votes
1answer
23 views

How do I resize output of a generative network to reconcile with the size of the real data?

I am designing a Generative Adversarial Network (GAN) trained on an image dataset. It has two components: the generator and the discriminator. The generative network outputs an artificial image. The ...
1
vote
2answers
171 views

What's the advantage of multi-scale architecture in Normalizing Flow?

While I've read a paper "Density Estimation using Real NVP", I have some confusing parts about multi-scale architecture in Section 3.6. 1. The author said that We implement a multi-scale ...
0
votes
1answer
30 views

Difference / Relationship of Generative Models / Variational Bayesian Inference

I feel a bit confused trying to merge and unify understandings of generative models and variational bayesian inference methods. Initially, I believed them to be the same thing, namely learning full ...
0
votes
0answers
23 views

Small handwriting database generation and augmentation

I am willing to extend my handwriting text recognition (HTR) offline system with some Spanish characters. So far, I have trained with images from the IAM Database, which includes english characters. ...
0
votes
0answers
19 views

Machine learning a discrete probability distribution that is parametrized by a set of real-valued parameters

Assume I have a probability distribution $p_\theta(\sigma)$ defined over discrete binary variables $\sigma$, $p_\theta(\sigma) : \sigma \to [0,1]$. This probability distribution is parametrized by a ...
0
votes
0answers
11 views

Generating Fixed Length Sequence with RNNs

Is there any way of generating fixed-length sequences with RNNs? I want to tell my character level RNN to generate a name of length 3, 4, 5 and so on. I haven't found anything online like this, but my ...

1
2 3 4 5