Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [generative-models]

A probabilistic or statistical model thought about as describing how the values in a sample is actually generated, and not only as a description or approximation.

1
vote
0answers
17 views
+50

Is the only difference between conditional generative models and discriminative models the complexity of the modeled distribution?

One common way to define discriminative models is that they model $P(Y|X)$, where $Y$ is the label, and $X$ is the observed variables. Conditional generative models do something quite similar, but the ...
1
vote
0answers
30 views

Why aren't auto-encoders also considered generative models?

Auto-encoders (AEs) are composed of an encoder and a decoder (often represented by a neural network). The encoder produces a vector representation $z$ of its input $x$ (e.g. an image). The decoder ...
0
votes
0answers
20 views

Uniform conjugate prior for a Beta distribution

Given $$ \pi \sim \text{Beta}(\alpha, \beta) $$ I'd like to place a prior on $\alpha$ and $\beta$. The "trick" mentioned in this post and this post seems to be to recognize that since $$ \begin{...
0
votes
0answers
11 views

Extracting likelihoods from generative model

I am looking for papers dealing with the extraction of explicit descriptions of probability distributions from a generative model. My use case is the following: I trained a GAN to generate samples ...
0
votes
0answers
9 views

RNN outputs noisy predictions

I have an RNN that I've trained and I'm now using to generate new sequences. These sequences are basically discrete state time courses for K different states. The ...
1
vote
0answers
10 views

How do I check my GAN implementation is correct?

I wrote a GAN implementation and I trained that to produce some sample images after training on a dataset. The images looked visually fine. Now I want to test my implementation on the CI and make ...
1
vote
1answer
17 views

Linking generative, discriminative models to supervised and unsupervised learning

Definitions that I am considering: A generative model learns p(x,y) whereas a discriminative model learns p(y|x=x). I would like to verify if my understanding is correct by sharing the following ...
0
votes
0answers
9 views

Can Conditional Random Fields(CRFs) become a discriminative model by changing partition function?

In Introduction to conditional random fields, page 24, equation (2.18) and (2.19), the linear-chain CRFs is defined as: $$p(y|x) = \frac{1}{Z(x)}\prod_{t=1}^{T}exp\{\sum_{k=1}^{K}\theta_kf_k(y_t,y_{t-...
0
votes
1answer
24 views

About discriminative/generative and directed/undirected graphical model?

I feel that most generative models happen to be DGM(directed graphical model), and most discriminative models are UGM(undirected graphical model). Is there any correlation between these concepts? ...
0
votes
0answers
16 views

Why is it easier to incorporate arbitrary features into discriminative models?

It is often stated, that when arbitrary features are implied, generative models (e.g. Naive Bayes) are a lesser fit than discriminative ones, mainly for being harder to build. How would you elucidate ...
1
vote
1answer
46 views

How can we relate the concepts of GAN/cGAN in SRGAN? Is SRGAN a Conditional GAN?

I have been reading and looking at implementations of the SRGAN, from "Photo-realistic Single Image Super Resolution with Generative Adversarial Networks" paper. One thing that I noticed is that the ...
0
votes
0answers
14 views

Architecture used by author in StackGAN

I was going through this paper stackedGAN I somehow understood how it is working. But I wanted to know it's architecture so that I can implement it myself. I went through the implementation code of ...
0
votes
3answers
112 views

What's the difference between estimating on a dataset $P(X|Y)$ and $P(Y)$ vs $P(Y|X)$? [closed]

In chapter 3 of his excellent book ("Generative and discriminative classifiers: Naive Bayes and logistic regression") , Tom Mitchell says that, when learning classifiers based on Bayes rule, one can ...
0
votes
0answers
21 views

Do generative models have less degrees of freedom than discriminant models?

I've read here that generative models have less degrees of freedom than discriminant ones, so they are more robust and less prone to overfitting. I would like to understand this statement with a ...
0
votes
1answer
31 views

On evaluating variational autoencoders with prior likelihood and reconstruction error

A common evaluation metric for variational autoencoders (VAEs) is estimating the marginal likelihood of some held-out data, i.e. $p(x)$. This is difficult and often one can only get a lower bound. It'...
0
votes
1answer
26 views

Other options for GAN losses

Would it be possible to update the loss of my generator using a loss function different from binary cross entropy? I have a multi-class labeled data-set and I want to train my discriminator to learn ...
0
votes
0answers
44 views

Shared latent spaces

I have two interrelated response variables $A$ and $B$ over each observation $i$ in my data. I am trying to create an unsupervised model where observations could be explained by means of latent spaces(...
0
votes
0answers
12 views

Does training a VAE online from a nonstationary distribution affect convergence?

For example, using data being sampled from reinforcement learning as the policy improves. If there is an issue, how would we address the issue?
0
votes
0answers
16 views

Wrong weights learned when training RBM

I'm training my RBM network and on epoch #4 I have such a filters representation (my weights matrix) But on the next iteration (fifth epoch) something went wrong and my filters became like this What ...
4
votes
1answer
32 views

Convergence to gradient in limit of variance

I came across this equation in the original GAN paper (pg 2 https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf): $$\lim_{\sigma \rightarrow 0} \nabla_{\bf x} \mathbb{E}_{\epsilon \sim \...
1
vote
0answers
32 views

Understanding probabilistic inference graphs

I am having trouble understanding inference graphs. In the diagram below I understand the graph on the left (forward graph) where the arrows describe the direction that data flows when training for ...
1
vote
0answers
22 views

Image generation based on sketch

Are there any instances of image generation models, where an image (a very rough sketch) has been used as an input and was then augmented. For example: This could be a rough sketch, which is then ...
2
votes
0answers
20 views

What is the difference between a conditional model and just having multiple models?

Say I have a labeled dataset that I want to create a generative model for, like a Generative Adversarial Network or a Variational Autoencoder. What do I gain or lose by making my models conditional (i....
0
votes
0answers
14 views

Applications of generative model except images, NLP and RL

Are there any other applications of generative model like VAE or GANs except image/video/NLP/RL? Particularly, are there any problems that are lower dimensional (3-100 dimensional) where generative ...
0
votes
1answer
37 views

Generative model to generate hidden activations coming from a previously trained hidden layer

I need to train a generative model to generate vectors which resemble the activations of a particular hidden layer of a neural network which has been previously trained. In particular, the hidden ...
0
votes
0answers
151 views

Which GAN is the best for data augmentation?

I have around 200000 images and I want to augment the data by generating more of them. Images do not have classes, because they are the same object and are used for the task of object detection. Can I ...
0
votes
0answers
17 views

Does maximizing model generativeness maximize its discriminative-ness?

If I train a model M in a way that maximizes the penalized likelihood of M given some data, is this equivalent to maximizing its ...
2
votes
1answer
43 views

Prerequisites for Wasserstein GAN/Autoencoder

Can someone who read WGAN/WAE papers and understood Wasserstein part, could you share how you prepared necessary Optimal Transport background? The mentioned papers seem little tough if you don't have ...
1
vote
0answers
16 views

GP-WGANs with Minibatch Discrimination

In the "Improved Training of Wasserstein GANs" paper the authors mentioned that batch normalization can not be used in combination with gradient penalty, since it introduces correlation between ...
9
votes
4answers
413 views

Likelihood-free inference - what does it mean?

Recently I have become aware of 'likelihood-free' methods being bandied about in literature. However I am not clear on what it means for an inference or optimization method to be likelihood-free. In ...
2
votes
0answers
149 views

Clarification: Are Generative Adversarial Networks an alternative to MCMC sampling?

I have been reading the original Goodfellow, et. al. paper on Generative Adversarial Networks and the way that they can obtain estimates of the posterior distribution of a discriminative network or ...
1
vote
1answer
107 views

Implementation of WAE-GAN does not match with the description in the paper

According to the litterature and specifically to this paper, the wasserstein autoencoders is an encoder-decoder architecture. So it must contain encoder and decoder parts. in the algorithm ...
0
votes
0answers
9 views

Sampling from factor analysis

Using the notation of Hinton and Ghahramani, the generative model a random vector $\textbf{x} \in \mathbb{R}^p$ under factor analysis is $$ \textbf{x} = \Lambda \textbf{z} + \textbf{u} \tag{1} $$ ...
0
votes
0answers
124 views

Generative Adversial Networks (GAN) - Dimension of the Latent Space

I am trying to synthesis medical images with GAN. The problem is that my generator loss is very bad behaved: I read that if latent space dimension is not enough for representation of the true ...
3
votes
1answer
136 views

Is the optimization of the Gaussian VAE well-posed?

In a Variational Autoencoder (VAE), given some data $x$ and latent variables $t$ with prior distribution $p(t) = \mathcal{N}(t \mid 0, I)$, the encoder aims to learn a distribution $q_{\phi}(t)$ that ...
0
votes
1answer
194 views

Discrete Random Variables and Deep Generative Models - Why Gumbel-Softmax is needed?

I am reading this 2014 NIPS paper on deep generative models and their application to latent discrete random variables, and this 2017 ICLR paper on Gumbel-Softmax. I essentially don't understand why we ...
1
vote
0answers
16 views

how to do convergence analysis for Generative adversarial network? [closed]

i have two variants of Generative adversarial networks. How to compare their performance with respect to converge?
0
votes
1answer
122 views

Defining ELBO in Variational Inference with 3 random variables

I am reading this paper, and having a hard time understanding one of the derivations. It is probably more of a stat question. The context is, having three random variables $x,y,z$, we would want to ...
0
votes
1answer
226 views

What kind of a generative model is an RNN?

Given the taxonomy of generative models as presented by Ian Goodfellow in Tutorial on Generative Adversarial Networks (https://arxiv.org/abs/1701.00160), in what branch do we put the family of RNNs?
0
votes
0answers
63 views

Variational Auto-encoder for supervised learning

It seems that variational auto-encoders (VAE) has become one of the most popular technique for generative modeling. However, is it possible to use variational auto encoders for discriminative ...
4
votes
1answer
256 views

When generating samples using variational autoencoder, we decode samples from $N(0,1)$ instead of $\mu + \sigma N(0,1)$

Context: I'm trying to understand the use of variational autoencoders as generators. My understanding: During training, for an input point $x_i$ we want to learn latent $\mu_i$ and $\sigma_i$ and ...
0
votes
0answers
115 views

Is there any Generative Model which can be used for Regression problems?

I've been researching Generative Models recently, and Probabilistic Graphical Models. Every time I read about Generative Models, I see they're trying to predict $P(x,y)$ or equivalently $P(x|y)$ and $...
7
votes
2answers
178 views

Is Reinforcement Learning the right choice for painting like Bob Ross?

My workplace is having a 2-week code challenge that involves producing an algorithm to reproduce 100 sample Bob Ross paintings as closely as possible given some constraints: "Paintings" are submitted ...
0
votes
0answers
220 views

GAN training: Both G and D has very low loss

I am training a GAN. At the beginning the generator has a very high loss, which converges over time. After some time, the image quality seems pretty good, but both the generator and discriminator have ...
3
votes
0answers
50 views

Generative Adversarial Network and Variational Autoencoders for Independent Component Analysis?

Background: I'm working on a model for independent component analysis (ICA) that is based on a methodology similar to GANs and VAEs. What I'm having trouble understanding is how the choice of the loss ...
2
votes
1answer
82 views

In Ian Goodfellow et al's paper “Generative Adversarial Networks”, why do they specify that they do not need a Markov chain or inference network?

In Ian Goodfellow et al's paper Generative Adversarial Networks, they state, "There is no need for any Markov chains or unrolled approximate inference networks during either training or generation of ...
3
votes
1answer
86 views

Getting more time series data

I am trying to do time series forecasting following this [1] paper. My data is represented by the total CPU load of a computer as my time series (one data point per second). Secondary features are ...
0
votes
2answers
127 views

Does generative model have to be machine learning-based?

According to Wikipedia, Given an observable variable X and a target variable Y, a generative model is a statistical model of the joint probability distribution on X × Y, P(X,Y) However, as far as ...
2
votes
1answer
246 views

Diversity of generated samples in VAE

In a variational autoencoder (VAE), it is possible to generate new samples (i.e. images) based on the latent space. After having read quite a few papers about VAE, I still wonder what drives the ...
0
votes
0answers
66 views

What's a good way to enforce shared latent space for multi-modal data

I have two separate data sets (of different types) $A$ and $B$. I can train two independent generative models (with different architectures for $A$ and $B$) such that $q_A(z|x)$ and $q_B(z|x)$ are the ...