Questions tagged [generative-models]

A probabilistic or statistical model thought about as describing how the values in a sample is actually generated, and not only as a description or approximation.

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

Generative capability of autoencoders and distribution modelling

I've been reading up quite a lot on autoencoders and variational inference. VAEs are used to generate data in accordance with the distribution of the training data. I am unable to understand how ...
1
vote
0answers
15 views

Why are autoregressive models neither directed or undirected, as described in the NADE paper?

In the paper Neural Autoregressive Distribution Estimation (Uria et al., 2016), NADE (and other autoregressive models) seem to be described as neither directed or undirected models: We’ve described ...
2
votes
0answers
25 views

Are there nonparametric generative models for datasets?

Typically when I see generative models, e.g., Latent Dirichlet Allocation (JMLR) or Linear/Quadratic Discriminant Analysis (wikipedia LDA), they are probabilistic models that belong to the exponential ...
4
votes
2answers
35 views

Is there a measure or notion of correlation (or association) without assuming finite second order moments?

When we study correlation or association in real data, do we always (implicitly) assume a finite second order moment for any hypothetical population distribution? If we do not assume this, what ...
1
vote
0answers
26 views

Controlling details in images generated by Generative Adversarial Networks (GANs)

With conditional GANs it is possible to generate images of a certain class of objects. And moreover with current text-to-image methods it seems to be possible to control certain details of the ...
0
votes
0answers
31 views

Generating Graphs using a Neural Network

I currently have constructed a Graph Neural Network in PyTorch with graph conv layers I have made. With this, I am able to feed in adjacency and feature matrices and successfully perform node ...
0
votes
0answers
23 views

Implementing the GAN loss function

I was reading the original Generative Adversarial Nets paper by Goodfellow et al. This is the minimax optimization problem for GANs: All the code I have seen, however, seem to have a different ...
0
votes
0answers
10 views

Why are Auxiliary Classifier GANs useful?

I am new to GANs and am learning about the different types of GANs. What added value do ACGANs provide over standard GANs and when should I use them? What is the difference conceptually between using ...
1
vote
1answer
21 views

Which is the decision rule in a gda classifier?

On the text book there is the following formula for the prediction rule, but I don’t understand where it comes from: The textbooks says it had been derived from I suppose pi and theta are the ...
3
votes
1answer
40 views

Generative design - algorithms and sources

I have recently encountered the term "generative design" where a computer algorithm (usually with the help of Machine Learning) comes up with new designs that conform to a certain set of requirements. ...
6
votes
2answers
87 views

What are the current methods to check for GAN overfitting?

In generative modeling, the goal is to find a way for a model to output samples of some distribution $p_X$ given a lot of samples $x_1, \ldots, x_n$. In particular, we want sampling from our model $G$ ...
0
votes
1answer
30 views

Basic doubt on generative models

I am new to statistics and while reading Bishop's book, in the 'Generative models' part 8.1.2. When explaining ancestral sample, he says: To do this, we start with the lowest-numbered node and ...
4
votes
3answers
328 views

What methods can be used for distribution generation other than GANs?

Generative Adversarial Networks (GANs) can be used for creating distributions of data points, that follow source data set distributions (e.g. images, sound, text, etc). Are there any other methods or ...
0
votes
0answers
14 views

Does discriminator converge to probability 0.5 in GAN just for the output of generator?

We know that in GANs, discriminator converges to 0.5. I want to become sure that the following statement is true: Discriminator converges to 0.5 for the examples generated by generator and for the ...
2
votes
2answers
25 views

How to extract crucial features to create an image

Imagine, you have a dataset containing pictures of (example only, just to explain the task) cats and dogs. The data set is labeled, so we can train using supervised learning algorithms. My goal is ...
7
votes
1answer
88 views

Structure of Generative Adversarial Networks (GAN) for mapping a simulation model

There is a simulation model of a system that I want to map as a neural network to test if a better execution time can be achieved with similar accuracy. The simulation model receives real-valued ...
0
votes
1answer
26 views

Reparametrization trick in VAE at Evaluation time

So I've been trying to implement the Variational Auto-Encoder model of Kigma et.al, but something has been bugging me. While I understand the need for reparametrization trick at training time, the ...
3
votes
1answer
47 views

Deep generative models learning a Bayesian-network distribution

Say I have a generative model for some distribution $p$ over a small number of RVs which allows me to easily sample from said distribution. For example, let's say it's a parameterized Bayesian network ...
2
votes
1answer
14 views

How to test generated text

I am creating a text generation algorithm for my master's research. I have a dialogue between two people and I would like to simulate one part of the conversation with naturally generated text (not ...
0
votes
0answers
18 views

How to draw CycleGANs discriminative distribution?

A few weeks ago I asked this question which was about how to understand GANs discriminative distribution. Now I am trying to draw a Figure like the one below, but for CycleGAN instead. In CycleGAN ...
0
votes
0answers
8 views

Generative Adversarial Networks evaluation methods for one channel

So I'm currently studying GANs with a focus on CycleGAN. I have trained my network on simulated images and real images. I did not train them as pairs but I have pairs available. The idea is now to ...
10
votes
1answer
489 views

How to understand Generative Adversarial Networks Discriminative distribution?

So I am currently studying Generative Adversarial Network and I read the paper by Goodfellow a few times now Generative Adversarial Nets and a few other papers in this field (DCGAN, CycleGAN, pix2pix, ...
1
vote
0answers
50 views

Are Flow Based Generative Models Referring to the Invertible Transformations?

So I have been studying generative models for a while now. I know how GANs and VAEs work quite well, but I am quite confused by how Flow Based Generative Models work. To my understanding, flow based ...
1
vote
0answers
19 views

Why generative models in Machine Learning are Boltzmann distribution-backed?

I learned from this review paper that MaxEnt models naturally display a Boltzmann distribution for the data samples, it comes from the Principle of Maximum Entropy. But I could not understand why this ...
0
votes
1answer
58 views

Weak Supervision - training generative model without knowing the true label

Recently I've been reading about weak supervision. I understand most of the concept details, there's one thing that is not clear to me though. In the generative model part (creating generative model ...
1
vote
1answer
39 views

How to generate random samples from a 2D dataset?

Suppose I'm given a data set consisting of many pairs of $(x,y)$ values which are correlated in some arbitrary complex way. How would I go about 'generating' more pairs of $(x,y)$ coordinates which ...
0
votes
1answer
50 views

Does variational auto-encoder output the variational distribution of the latent variable or the distribution of the input x?

In the simple case of mixture of gaussians(with known variance), we have 2 latent variables $\mu$ and $z$. In the vaiational auto-encoder, we assume that the model is infinite mixture of gaussians. If ...
0
votes
1answer
51 views

Learning Deep Generative Models of Graphs

I'm reading through Learning Deep Generative Models of Graphs, which is a paper that seems to me propose some sort of variational autoencoder to generate a graph. At very high level the semantic of ...
0
votes
1answer
75 views

Maximum likelihood solution in probabilistic generative models

I am reading Bishop's "Machine Learning and Pattern Recognition" (https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf) and I have the ...
1
vote
1answer
81 views

Looking at training progress of generative adversarial network (GAN) - what to look for?

I'm trying to understand the output of a GAN training process. I can always inspect the visual output (the generated images) to see how the GAN's evolving, but how can I evaluate training based on the ...
1
vote
1answer
27 views

Is it possible to obtain learn specific information about a distribution from a GAN?

If I feed a GAN some images of Gaussian noise with some $\sigma$ and it is successful at generating similar images, is there some way to recover $\sigma$ from the gan or is the generator purely a ...
1
vote
0answers
32 views

Discriminative Models with Class Priors

In discriminative models, we model $p(Y|X)$ directly while in generative models we model $p(X|Y)p(Y)$ where $X$ is the input and $Y$ is the output variable. I am confused when the parameters and ...
1
vote
0answers
14 views

Why is the latent loss in VAEs is set to the KL divergence?

The KL Divergence is surely not the wrong way to go, but I wonder if there are any VAEs which use something like the Wasserstein Distance or even an l2 loss on the ...
1
vote
2answers
30 views

Working mechanism of discriminator in text to image synthesis GAN

I have the following architecture of discriminator in text to image synthesis where the image is convolved to lower dimension and concatenated with the text . My question is what is the use of ...
2
votes
1answer
132 views

How does one generate (smooth) varying size output signals with Machine Learning?

I am interested in knowing about generative methods that generate signals (e.g. images) of varying sizes. But the size generation being sort of "smooth/continuous". So for example, generating images ...
2
votes
0answers
18 views

Generating distributions with very small values with generative adverserial networks

I am using a GAN to generate a vector distribution. However, most of the values of this distribution are very close to zero, i.e $1 \times 10^{-7}$. The distribution also have minus values. With ...
1
vote
0answers
113 views

How to use KL divergence to compare two distributions?

I am trying to model the probability distribution of a multi-dimensional dataset where all the values are discrete. Suppose the training data (represented by T) is of the shape (m, n) where n is the ...
2
votes
0answers
249 views

GANs for non image data

I'm looking to narrow down the subject for my bachelor thesis: I am currently working on a project, that only offers a small dataset and there will be no more data incoming for now. What I'm trying to ...
0
votes
2answers
97 views

Why can logistic regression always outperform naive Bayes for every conditional distribution?

We know that the generative model assumes that $X_i \perp X_{-i}| Y$; while the discriminative model assumes that $p(Y=1|x; \alpha)=\frac{e^{\alpha_0+\sum_{i=1}^n\alpha_ix_i}}{1+e^{\alpha_0+\sum_{i=1}^...
0
votes
0answers
320 views

Linear discriminant analysis- generative or discriminative

According to this link LDA is a generative classifier. But the name itself has got the word 'discriminant'. Also, the motto of LDA is to model a discriminant function to classify. Then why is this a ...
5
votes
0answers
192 views

Is the only difference between conditional generative models and discriminative models the complexity of the modeled distribution?

One common way to define discriminative models is that they model $P(Y|X)$, where $Y$ is the label, and $X$ is the observed variables. Conditional generative models do something quite similar, but the ...
2
votes
1answer
114 views

Why aren't auto-encoders also considered generative models?

Auto-encoders (AEs) are composed of an encoder and a decoder (often represented by a neural network). The encoder produces a vector representation $z$ of its input $x$ (e.g. an image). The decoder ...
0
votes
0answers
38 views

Uniform conjugate prior for a Beta distribution

Given $$ \pi \sim \text{Beta}(\alpha, \beta) $$ I'd like to place a prior on $\alpha$ and $\beta$. The "trick" mentioned in this post and this post seems to be to recognize that since $$ \begin{...
1
vote
0answers
26 views

Extracting likelihoods from generative model

I am looking for papers dealing with the extraction of explicit descriptions of probability distributions from a generative model. My use case is the following: I trained a GAN to generate samples ...
0
votes
0answers
39 views

RNN outputs noisy predictions

I have an RNN that I've trained and I'm now using to generate new sequences. These sequences are basically discrete state time courses for K different states. The ...
2
votes
0answers
23 views

How do I check my GAN implementation is correct?

I wrote a GAN implementation and I trained that to produce some sample images after training on a dataset. The images looked visually fine. Now I want to test my implementation on the CI and make ...
3
votes
2answers
131 views

Linking generative, discriminative models to supervised and unsupervised learning

Definitions that I am considering: A generative model learns p(x,y) whereas a discriminative model learns p(y|x=x). I would like to verify if my understanding is correct by sharing the following ...
0
votes
0answers
21 views

Can Conditional Random Fields(CRFs) become a discriminative model by changing partition function?

In Introduction to conditional random fields, page 24, equation (2.18) and (2.19), the linear-chain CRFs is defined as: $$p(y|x) = \frac{1}{Z(x)}\prod_{t=1}^{T}exp\{\sum_{k=1}^{K}\theta_kf_k(y_t,y_{t-...
1
vote
1answer
111 views

What is the relationship between discriminative and generative and directed and undirected graphical models?

I feel that most generative models happen to be DGM(directed graphical model), and most discriminative models are UGM(undirected graphical model). Is there any correlation between these concepts? ...
0
votes
0answers
17 views

Why is it easier to incorporate arbitrary features into discriminative models?

It is often stated, that when arbitrary features are implied, generative models (e.g. Naive Bayes) are a lesser fit than discriminative ones, mainly for being harder to build. How would you elucidate ...