Questions tagged [generative-models]

A probabilistic or statistical model thought about as describing how the values in a sample is actually generated, and not only as a description or approximation.

Filter by
Sorted by
Tagged with
1
vote
0answers
12 views

How can I construct a classifier two-sample test to approximately distinguish between two distributions?

Classifier two-sample tests (C2STs; e.g., as described by Lopez-Paz and Oquab, 2017) aim to test whether two samples $\mathbf{X} \sim P$ and $\mathbf{Y} \sim Q$ come from the same distribution (i.e., ...
0
votes
0answers
6 views

Normalizing Flows notations and change of variable formula

Maybe it is a silly question but I am wondering why in some paper the function $f$ is the mapping from $x$ to $z$ and on some others it is the other way around. Typically in this review we go from $x$ ...
2
votes
1answer
24 views

Can a Simple ANN be Generative?

If a simple ANN was trained to predict the next step in a sequence, such as a univariate time series, can it be considered a generative model?
1
vote
1answer
95 views

Are text generation models generative or discriminative?

I've recently been studying generative and discriminative models, and I had a question regarding text generation. I'm aware that generative models model $P(X, Y)$ and discriminative models model $P(Y |...
0
votes
1answer
30 views

How to find bits/dim of a gaussian output distribution?

I have images that are 64x64x3 and 64x64x1 8-bit. I transform those images down to [-1,1]. I now want to find the bits/dim for my VAE log probability. How do I find the bits/dim of the log likelihood? ...
2
votes
1answer
10 views

Updating of characer's embedding inside an RNN during character generation task

In one of the tutorials of tensorflow, there is a "text generation with an RNN" tutorial. When creating the model, they create a mapping of characters to IDs and vice versa. Then in the ...
0
votes
0answers
8 views

Difference between Structural Topic Modeling(STM) and SAGE (Sparse Additive Generative Model)?

I have read that STM combines 3 models of: (1) correlated topic model (CTM) (2) Dirichlet-Multinomial Regression (DMR) topic model (3) Sparse Additive Generative Model (SAGE) Is it correct to just ...
1
vote
1answer
34 views

When is my Wasserstein GAN-GP overfitting?

I have a hard time interpreting the WGAN-GP losses attached. At which epoch is D and/or G overfitting? The quality improves a lot overtime, yet the generator loss at later epochs does not appear on ...
0
votes
0answers
13 views

Projection pursuit optimal hidden units RBM

I am training a Restricted Boltzmann machine. I gather from this presentation: https://cseweb.ucsd.edu/~dasgupta/254-deep/nakul.pdf that projection pursuit can be used for density estimation of the ...
0
votes
0answers
11 views

Why does using conditional random field avoid independence assumption

I am reading Daphne Koller's book on probabilistic graphical models under the topic of conditional random fields. One of the advantages in using CRF is that we can avoid modelling the correlations ...
0
votes
0answers
23 views

Wrong reconstruction with Wasserstein loss conditional GAN with gradient penalization (WcGAN-GP)

I have a following dataset of $(X_\mathrm{real}, y_\mathrm{real})$ sample/label pairs, where both $X_\mathrm{real}$ and $y_\mathrm{real}$ are multidimensional vectors. I'm trying to build the ...
1
vote
1answer
36 views

Latent Dirichlet Allocation - dimensionality of the Dirichlet prior parameter

I seeking some clarity on the dimensionality of the (hyper)parameter $\eta$ of the "smoothed LDA" model in Section 5.4 of the original paper by Blei, Ng, Jordan (2003), which can be found ...
0
votes
0answers
6 views

Network structure of generative model for classification

I'm trying to model a generative model for classification problem, especially aiming to solve an imbalanced data problem. However, I couldn't get intuitive understanding for generative classifier in ...
0
votes
0answers
32 views

Is it wrong to use tanh with images normalized in [0,1] range?

I've seen in some repositories, mostly related to GANs (Generative Adversarial Networks) using tanh activation function whilst having input images in the range of <...
-1
votes
1answer
20 views

K-Lipschitz for the Wasserstein GANs

I am trying to follow this blog for Wasserstein loss for Generative Adversarial Networks: From GAN to WGAN. Actually, I am trying to follow the logic behind the K-Lipschitz continuity. This post in ...
1
vote
1answer
246 views

Why use tanh function at the last layer of generator in GAN?

While studying GAN, I found out that ReLU activation is used at the intermediate layers, and tanh or sigmoid is used at the last layer of the generator. I'm curious about why sigmoid or tanh is used ...
0
votes
0answers
11 views

Help in calculating diagonal covariance matrix for generative model for binary classification

I am given this data. I want to fit a generative model $\cal{N}(\mu_0, \sigma_0^2 I_2)$, $\cal{N}(\mu_1, \sigma_1^2 I_2)$ for the $0$ and $1$ classes respectively using $\textbf{MLE}$ and plot ...
0
votes
1answer
31 views

How do I revert a probability distribution back to its source data samples?

Normally we first collect real samples into a dataset and describe its probability distribution parametrically or empirically. If I instead generate a parametric distribution for artificial data ...
1
vote
1answer
61 views

Modern applications of Bayesian Model Selection

I'm trying to understand the merits of this field so I'll try to break down my question. Research: Is Bayesian model selection considered a popular topic of research these days? Variable selection: ...
1
vote
0answers
13 views

entropy regularization in generative model

I am wondering if it is possible to use entropy as a regularization in a generative model. For example, in the conjugate model where $x_i \in X$ is observed data and generated from a Normal ...
2
votes
1answer
74 views

Inference in Normalizing Flow model: NICE(non linear independent components estimation)

I was recently reading Y. Bengio's paper on NICE (https://arxiv.org/abs/1410.8516). In the paper, authors have taken a view that a good representation involves easy learning of the data distribution. ...
2
votes
0answers
53 views

Can PCA generate a new random image?

I read The Batch: GANs newsletter and Goodfellow said: My colleague Bing Xu modeled face images from the Toronto Face Database, which were only 90 pixels square and grayscale. Because the faces were ...
1
vote
0answers
35 views

Probabilistic Difference between Autoencoders and Variational Autoencoders

I have recently read up about Autoencoders and Variational Autoencoders. In Variational Autoencoders, the loss is modeled based on what distribution we choose for P(x|z). So, if we choose it to be ...
0
votes
0answers
17 views

Probabilistic generative models for clustering and classification

I have a question regarding the probabilistic setting of clustering and classification. More specifically regarding Gaussian Mixture Models and probabilistic generative models for classification. In ...
0
votes
1answer
83 views

Why VAE need reparameterization trick while LDA does not (both using variational inference for optimization)

Both VAE and LDA (latent Dirichlet allocation) is based on variational inference, and they both try to optimize ELBO objective function Variational autoencoders use reparameterization so that "...
0
votes
0answers
43 views

Variational Autoencoder with a Flexible Prior

Let's say I have a Variational autoencoder with a Gaussian prior architecture and I want to regularize this VAE with a flexible prior. Does sampling with the normal "reparameterization trick"...
5
votes
1answer
145 views

How well does GAN (generative adversarial network) perform for small samples?

GAN is an unsupervised learning algorithm that pits a discriminator and generator against one another so that they iteratively compete to enhance the overall model's ability to model/replicate a given ...
0
votes
2answers
179 views

In VAE, why use MSE loss between input x and decoded sample x' from latent distribution?

Variational Autoencoders (VAEs) are based on the concept of Variational Inference (VI) and use two Neural Networks similar to Vanilla Autoencoders (AEs) for function approximation. I understood the ...
1
vote
1answer
245 views

Why we learn $\log{\sigma^2}$ in VAE reparameterization trick instead of standard deviation?

We know that the reparameterization trick is to learn two vectors $\sigma$ and $\mu$, sample $\epsilon$ from $N(0, 1)$ and then your latent vector $Z$ would be (where $\odot$ is the element-wise ...
5
votes
1answer
177 views

reparameterization trick in VAEs, How should we do this?

I'm confused about how does reparameterization trick works. In this article shows it very simple. You learn two vectors $\sigma$ and $\mu$, sample $\epsilon$ from $N(0, 1)$ and then your latent ...
4
votes
2answers
161 views

Variational Autoencoder (VAE) latent features

I'm new to DL and I'm working on VAE for biomedical images. I need to extract relevant features from ct scan. So I created first an autoencoder and after a VAE. My doubt is that I don't know from ...
0
votes
1answer
162 views

What is the Intuition behind the GAN Discriminator loss? How does Discriminator loss works?

I have just stated learning GAN and the loss used are different for same problems in same tutorial. Could someone please tell me intutively that which loss function is doing what? For example, in the ...
1
vote
2answers
635 views

What is the stop criteria of generative adversarial nets?

I have used the GANs (Generative Adversarial Networks) with a binary cross-entropy loss function (in both generator and discriminator). Throughout the training step, the variation of generator loss ...
0
votes
0answers
23 views

Conditional generative models for large numbers of classes

I'm looking for a conditional generative model — could be cGAN or cVAE — that is particularly well suited to large numbers of classes. I'm expecting to require 500+ classes, but so many models are ...
2
votes
1answer
130 views

Generating distributions with with given variance, skewness, and kurtosis

I would like to generate distributions that are as close to normal as possible, except for the deviations shows below. The options I've located have properties that are related to the families of ...
0
votes
1answer
17 views

Generating text using LSTM given condition vector

I know that you can use an RNN to generate text given the first few letters ...
0
votes
0answers
7 views

the IS(Inception Score) of the generated images is higher than the images from datasets?

Is there a situation where the IS(Inception Score) of the generated images is higher than the images from datasets? Can anyone tell me?
2
votes
0answers
17 views

Softmax classifier with class priors

Softmax classifier is a discriminative model that directly models $p(Y|X, w)$ where $Y$ is the label for input $X$. We can write it as follows: $$p(y_i=k|x_i,w_k)=\frac{\exp \lbrace w_k^T x_i \rbrace}{...
2
votes
0answers
32 views

Extremely basic question: how are data assumed to be generated in machine learning?

Given a dataset $\mathcal{D} = \{x_i\}, i = 1, \ldots, N, x_i \in \mathbb{R}$ In machine learning, what assumption is made as to how data are generated? I've seen two basic ideas circulating around, ...
1
vote
1answer
16 views

How do we generate the samples of hidden root nodes in the Bayes network (Sigmoid Belief Networks) of a generative model

Following is a Sigmoid Belief Networks where we can only observe the bottom observable layer $v.$ Usually we use ...
0
votes
1answer
62 views

Distribution of Data in GAN

Will Generator and Discriminator learns the Distribution of Data when it was trained? If So Both Generator and Discriminator learns same distribution or different distributions ? What can we interpret ...
0
votes
0answers
22 views

proof of For $G$ fixed, the optimal discriminator $D$

This question is related to a part of Why Discriminator converges to 1/2 in Generative Adversarial Networks? I'm looking at the proof of For $G$ fixed, the optimal discriminator $D$ is $ D_{G}^{*}(\...
1
vote
0answers
37 views

LSTM high accuracy but poor generation performance

I'm writing a LSTM model for generating music (in particular drums). My model is based on these 2 models: LSTM text generator LSTM drum generator The model seems to work fine, it trains and I can ...
0
votes
0answers
12 views

Can Gaussian mixture models help an algorithm target a specific cluster?

In the chart below is a Gaussian Mixture model (GMM) based on three time series or datasets that the model was able to easily cluster into three different colored classes. Class 1 is the blue ellipse ...
2
votes
1answer
440 views

How to interprete Discriminator and Generator loss in GAN

I trained GAN with learning rate 0.00002, discriminator is trained once and generator is trained twice per epoch. Wasserstein loss is used as loss function This is the loss graph for discriminator ...
1
vote
0answers
27 views

What are the best known techniques to verify that a GAN samples correctly from a given distribution?

I would like to know what are the best known techniques to check that a generative adversarial network (GAN) samples from the correct distribution. Naively, I would say it all boils down to a ...
0
votes
1answer
92 views

Understanding reparameterization trick and training process in variational autoencoders

I am trying to understand variational autoencoders, particularly the sampling component and the reparameterization trick. I understand that instead of using a fixed determinstic latent representation ...
1
vote
0answers
170 views

Uniformly distributed VAE samples

I am currently working on a VAE to generate images (for simplicity MNIST). If I understand the theory correctly, the latent variables follow a gaussian normal distribution in the dimensions of the ...
1
vote
0answers
31 views

What is the intution behind generative and discriminative models?

Lets say we have a dataset $D=(X,Y)$ and our aim is to find a model $f$ which maps our feature and target. $f:X \rightarrow Y$ According to wikipedia, A discriminative model is a model of the ...
0
votes
0answers
72 views

Optimal critic in Wasserstein GAN (WGAN)

In the Wasserstein GAN (generative adversarial network) paper they say that the critic needs to be optimal. What does this mean practically? They do more training iterations of critic (5) than the ...

1
2 3 4 5