Questions tagged [generative-models]

A probabilistic or statistical model thought about as describing how the values in a sample is actually generated, and not only as a description or approximation.

Filter by
Sorted by
Tagged with
0
votes
0answers
26 views

Weak Supervision - training generative model without knowing the true label

Recently I've been reading about weak supervision. I understand most of the concept details, there's one thing that is not clear to me though. In the generative model part (creating generative model ...
1
vote
1answer
31 views

How to generate random samples from a 2D dataset?

Suppose I'm given a data set consisting of many pairs of $(x,y)$ values which are correlated in some arbitrary complex way. How would I go about 'generating' more pairs of $(x,y)$ coordinates which ...
0
votes
1answer
24 views

Does variational auto-encoder output the variational distribution of the latent variable or the distribution of the input x?

In the simple case of mixture of gaussians(with known variance), we have 2 latent variables $\mu$ and $z$. In the vaiational auto-encoder, we assume that the model is infinite mixture of gaussians. If ...
0
votes
1answer
40 views

Learning Deep Generative Models of Graphs

I'm reading through Learning Deep Generative Models of Graphs, which is a paper that seems to me propose some sort of variational autoencoder to generate a graph. At very high level the semantic of ...
0
votes
1answer
54 views

Maximum likelihood solution in probabilistic generative models

I am reading Bishop's "Machine Learning and Pattern Recognition" (https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf) and I have the ...
0
votes
1answer
20 views

Looking at training progress of generative adversarial network (GAN) - what to look for?

I'm trying to understand the output of a GAN training process. I can always inspect the visual output (the generated images) to see how the GAN's evolving, but how can I evaluate training based on the ...
1
vote
1answer
24 views

Is it possible to obtain learn specific information about a distribution from a GAN?

If I feed a GAN some images of Gaussian noise with some $\sigma$ and it is successful at generating similar images, is there some way to recover $\sigma$ from the gan or is the generator purely a ...
1
vote
0answers
25 views

Discriminative Models with Class Priors

In discriminative models, we model $p(Y|X)$ directly while in generative models we model $p(X|Y)p(Y)$ where $X$ is the input and $Y$ is the output variable. I am confused when the parameters and ...
1
vote
0answers
13 views

Why is the latent loss in VAEs is set to the KL divergence?

The KL Divergence is surely not the wrong way to go, but I wonder if there are any VAEs which use something like the Wasserstein Distance or even an l2 loss on the ...
1
vote
2answers
20 views

Working mechanism of discriminator in text to image synthesis GAN

I have the following architecture of discriminator in text to image synthesis where the image is convolved to lower dimension and concatenated with the text . My question is what is the use of ...
2
votes
1answer
79 views

How does one generate (smooth) varying size output signals with Machine Learning?

I am interested in knowing about generative methods that generate signals (e.g. images) of varying sizes. But the size generation being sort of "smooth/continuous". So for example, generating images ...
1
vote
0answers
17 views

Generating distributions with very small values with generative adverserial networks

I am using a GAN to generate a vector distribution. However, most of the values of this distribution are very close to zero, i.e $1 \times 10^{-7}$. The distribution also have minus values. With ...
1
vote
0answers
49 views

How to use KL divergence to compare two distributions?

I am trying to model the probability distribution of a multi-dimensional dataset where all the values are discrete. Suppose the training data (represented by T) is of the shape (m, n) where n is the ...
1
vote
0answers
82 views

GANs for non image data

I'm looking to narrow down the subject for my bachelor thesis: I am currently working on a project, that only offers a small dataset and there will be no more data incoming for now. What I'm trying to ...
0
votes
0answers
23 views

Why can logistic regression always outperform naive Bayes for every conditional distribution?

We know that the generative model assumes that $X_i \perp X_{-i}| Y$; while the discriminative model assumes that $p(Y=1|x; \alpha)=\frac{e^{\alpha_0+\sum_{i=1}^n\alpha_ix_i}}{1+e^{\alpha_0+\sum_{i=1}^...
0
votes
0answers
67 views

Linear discriminant analysis- generative or discriminative

According to this link LDA is a generative classifier. But the name itself has got the word 'discriminant'. Also, the motto of LDA is to model a discriminant function to classify. Then why is this a ...
2
votes
0answers
100 views

Is the only difference between conditional generative models and discriminative models the complexity of the modeled distribution?

One common way to define discriminative models is that they model $P(Y|X)$, where $Y$ is the label, and $X$ is the observed variables. Conditional generative models do something quite similar, but the ...
2
votes
1answer
82 views

Why aren't auto-encoders also considered generative models?

Auto-encoders (AEs) are composed of an encoder and a decoder (often represented by a neural network). The encoder produces a vector representation $z$ of its input $x$ (e.g. an image). The decoder ...
0
votes
0answers
24 views

Uniform conjugate prior for a Beta distribution

Given $$ \pi \sim \text{Beta}(\alpha, \beta) $$ I'd like to place a prior on $\alpha$ and $\beta$. The "trick" mentioned in this post and this post seems to be to recognize that since $$ \begin{...
0
votes
0answers
22 views

Extracting likelihoods from generative model

I am looking for papers dealing with the extraction of explicit descriptions of probability distributions from a generative model. My use case is the following: I trained a GAN to generate samples ...
0
votes
0answers
18 views

RNN outputs noisy predictions

I have an RNN that I've trained and I'm now using to generate new sequences. These sequences are basically discrete state time courses for K different states. The ...
1
vote
0answers
19 views

How do I check my GAN implementation is correct?

I wrote a GAN implementation and I trained that to produce some sample images after training on a dataset. The images looked visually fine. Now I want to test my implementation on the CI and make ...
1
vote
1answer
43 views

Linking generative, discriminative models to supervised and unsupervised learning

Definitions that I am considering: A generative model learns p(x,y) whereas a discriminative model learns p(y|x=x). I would like to verify if my understanding is correct by sharing the following ...
0
votes
0answers
15 views

Can Conditional Random Fields(CRFs) become a discriminative model by changing partition function?

In Introduction to conditional random fields, page 24, equation (2.18) and (2.19), the linear-chain CRFs is defined as: $$p(y|x) = \frac{1}{Z(x)}\prod_{t=1}^{T}exp\{\sum_{k=1}^{K}\theta_kf_k(y_t,y_{t-...
1
vote
1answer
65 views

What is the relationship between discriminative and generative and directed and undirected graphical models?

I feel that most generative models happen to be DGM(directed graphical model), and most discriminative models are UGM(undirected graphical model). Is there any correlation between these concepts? ...
0
votes
0answers
17 views

Why is it easier to incorporate arbitrary features into discriminative models?

It is often stated, that when arbitrary features are implied, generative models (e.g. Naive Bayes) are a lesser fit than discriminative ones, mainly for being harder to build. How would you elucidate ...
1
vote
1answer
146 views

How can we relate the concepts of GAN/cGAN in SRGAN? Is SRGAN a Conditional GAN?

I have been reading and looking at implementations of the SRGAN, from "Photo-realistic Single Image Super Resolution with Generative Adversarial Networks" paper. One thing that I noticed is that the ...
0
votes
0answers
16 views

Architecture used by author in StackGAN

I was going through this paper stackedGAN I somehow understood how it is working. But I wanted to know it's architecture so that I can implement it myself. I went through the implementation code of ...
0
votes
3answers
114 views

What's the difference between estimating on a dataset $P(X|Y)$ and $P(Y)$ vs $P(Y|X)$? [closed]

In chapter 3 of his excellent book ("Generative and discriminative classifiers: Naive Bayes and logistic regression") , Tom Mitchell says that, when learning classifiers based on Bayes rule, one can ...
0
votes
0answers
28 views

Do generative models have less degrees of freedom than discriminant models?

I've read here that generative models have less degrees of freedom than discriminant ones, so they are more robust and less prone to overfitting. I would like to understand this statement with a ...
0
votes
1answer
79 views

On evaluating variational autoencoders with prior likelihood and reconstruction error

A common evaluation metric for variational autoencoders (VAEs) is estimating the marginal likelihood of some held-out data, i.e. $p(x)$. This is difficult and often one can only get a lower bound. It'...
0
votes
1answer
34 views

Other options for GAN losses

Would it be possible to update the loss of my generator using a loss function different from binary cross entropy? I have a multi-class labeled data-set and I want to train my discriminator to learn ...
0
votes
0answers
75 views

Shared latent spaces

I have two interrelated response variables $A$ and $B$ over each observation $i$ in my data. I am trying to create an unsupervised model where observations could be explained by means of latent spaces(...
0
votes
0answers
21 views

Does training a VAE online from a nonstationary distribution affect convergence?

For example, using data being sampled from reinforcement learning as the policy improves. If there is an issue, how would we address the issue?
0
votes
0answers
22 views

Wrong weights learned when training RBM

I'm training my RBM network and on epoch #4 I have such a filters representation (my weights matrix) But on the next iteration (fifth epoch) something went wrong and my filters became like this What ...
4
votes
1answer
45 views

Convergence to gradient in limit of variance

I came across this equation in the original GAN paper (pg 2 https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf): $$\lim_{\sigma \rightarrow 0} \nabla_{\bf x} \mathbb{E}_{\epsilon \sim \...
1
vote
0answers
37 views

Understanding probabilistic inference graphs

I am having trouble understanding inference graphs. In the diagram below I understand the graph on the left (forward graph) where the arrows describe the direction that data flows when training for ...
1
vote
0answers
27 views

Image generation based on sketch

Are there any instances of image generation models, where an image (a very rough sketch) has been used as an input and was then augmented. For example: This could be a rough sketch, which is then ...
2
votes
0answers
33 views

What is the difference between a conditional model and just having multiple models?

Say I have a labeled dataset that I want to create a generative model for, like a Generative Adversarial Network or a Variational Autoencoder. What do I gain or lose by making my models conditional (i....
0
votes
0answers
18 views

Applications of generative model except images, NLP and RL

Are there any other applications of generative model like VAE or GANs except image/video/NLP/RL? Particularly, are there any problems that are lower dimensional (3-100 dimensional) where generative ...
0
votes
1answer
40 views

Generative model to generate hidden activations coming from a previously trained hidden layer

I need to train a generative model to generate vectors which resemble the activations of a particular hidden layer of a neural network which has been previously trained. In particular, the hidden ...
0
votes
0answers
199 views

Which GAN is the best for data augmentation?

I have around 200000 images and I want to augment the data by generating more of them. Images do not have classes, because they are the same object and are used for the task of object detection. Can I ...
0
votes
0answers
30 views

Does maximizing model generativeness maximize its discriminative-ness?

If I train a model M in a way that maximizes the penalized likelihood of M given some data, is this equivalent to maximizing its ...
2
votes
1answer
63 views

Prerequisites for Wasserstein GAN/Autoencoder

Can someone who read WGAN/WAE papers and understood Wasserstein part, could you share how you prepared necessary Optimal Transport background? The mentioned papers seem little tough if you don't have ...
1
vote
0answers
24 views

GP-WGANs with Minibatch Discrimination

In the "Improved Training of Wasserstein GANs" paper the authors mentioned that batch normalization can not be used in combination with gradient penalty, since it introduces correlation between ...
11
votes
4answers
910 views

Likelihood-free inference - what does it mean?

Recently I have become aware of 'likelihood-free' methods being bandied about in literature. However I am not clear on what it means for an inference or optimization method to be likelihood-free. In ...
2
votes
0answers
228 views

Clarification: Are Generative Adversarial Networks an alternative to MCMC sampling?

I have been reading the original Goodfellow, et. al. paper on Generative Adversarial Networks and the way that they can obtain estimates of the posterior distribution of a discriminative network or ...
1
vote
1answer
153 views

Implementation of WAE-GAN does not match with the description in the paper

According to the litterature and specifically to this paper, the wasserstein autoencoders is an encoder-decoder architecture. So it must contain encoder and decoder parts. in the algorithm ...
0
votes
0answers
10 views

Sampling from factor analysis

Using the notation of Hinton and Ghahramani, the generative model a random vector $\textbf{x} \in \mathbb{R}^p$ under factor analysis is $$ \textbf{x} = \Lambda \textbf{z} + \textbf{u} \tag{1} $$ ...
0
votes
0answers
138 views

Generative Adversial Networks (GAN) - Dimension of the Latent Space

I am trying to synthesis medical images with GAN. The problem is that my generator loss is very bad behaved: I read that if latent space dimension is not enough for representation of the true ...