Questions tagged [generative-models]

A probabilistic or statistical model thought about as describing how the values in a sample is actually generated, and not only as a description or approximation.

Filter by
Sorted by
Tagged with
0 votes
0 answers
20 views

My WGAN-GP isn't capturing bi-modal distributed underlying data

I am training a WGAN to generate a vector of 6 data points (6 dimensional output). I plotted a correlation matrix of my dataset to see the underlying distribution of the 6 data points, and the ...
ShrHon's user avatar
  • 1
2 votes
0 answers
38 views

Maximum Mean Discrepancy (MMD) implementation as a metric to measure GAN performance

I am trying to evaluate the performance of the GAN model, I have trained. I found that there exist two major choices FID (Fréchet inception distance) and MMD (Maximum Mean Discrepancy) for comparing ...
Rajesh Nakka's user avatar
1 vote
0 answers
10 views

Help finding Bayesian model with multi-modal posterior [closed]

Background: There is a paper (link) that concerns combining MCMC methods with a normalizing flow (a type of generative model). The basic idea is that the normalizing flow helps propose samples, which ...
caitlin's user avatar
  • 31
0 votes
0 answers
48 views

Confused on Kullback-Leibler divergence being invoked without proper definition

I am trying to understand how authors of the DDPM paper in appendix A, made the leap from equation 21 to equation 22. Specifically, it is not clear to me how they managed to convert the first term of ...
Spacey's user avatar
  • 1,805
0 votes
0 answers
11 views

Can inverse sampling method be adapted to random vectors?

This might be a very basic question, but it seems that in all the examples I've seen, the inverse sampling method (i.e., input uniform RV into the inverse of CDF of desired PDF/probability ...
Fraïssé's user avatar
  • 1,520
1 vote
0 answers
27 views

How does the order of implementation of Generative Data Augmentation and Generative Audio Super-Resoln matter? Which one should be implemented first?

I have a low quality audio dataset that I will use for classification. My goal is both to increase the quality of this dataset to make it easier to label it with supervised methods (super resolution) ...
Yalçın Cenik's user avatar
0 votes
0 answers
18 views

Encoder-decoder Transformer model makes outputs predictions almost perfectly but fails to autoregressively decode

The model's sample predictions that I'm printing during training are almost perfect but the model generates meaningless tokens during evaluation. For training I'm feeding it the source and target ...
Sean's user avatar
  • 3,767
0 votes
0 answers
20 views

VAE for Motion Sequence Generation - Convergence Issue with Scheduled Sampling

I implemented a Variational Autoencoder (VAE) in PyTorch for motion sequence generation using human pose data (joint angles and angular velocities in radians) from the CMU dataset. The VAE ...
RTn's user avatar
  • 1
0 votes
0 answers
351 views

How to calculate FID for a set with a small number of images?

I need to evaluate my generative model using FID (Fréchet inception distance). However, the dataset of real images that I have only contains 2719 examples. I've read that the authors of the metric ...
nietoperz21's user avatar
0 votes
0 answers
20 views

Should I account for email opens when modeling link clicks embedden in the email?

I'm trying to build a model that predicts whether an email recipient will click on a link inside an email. As input, it takes member-related email click/open/unsubscribe history, as well as that ...
Jake's user avatar
  • 199
0 votes
1 answer
131 views

Why does higher dimensional data has higher likelihood?

I am reading about generative models. I came across an example a few times but I cannot come up with an explanation for it. Imagine data is generated according to $p_\text{data}(x)$. It is often said ...
Alf's user avatar
  • 77
1 vote
0 answers
45 views

Why are LLMs generative models [duplicate]

According to Wikipedia: A generative model is a statistical model of the joint probability distribution $P ( X , Y )$ on given observable variable $X$ and target variable $Y$; A discriminative model ...
Sam's user avatar
  • 363
1 vote
1 answer
31 views

In what sense do Bayesian models simulate observation?

Could you verify that i understand the author correctly here? New to Bayes. [...] These assumptions [the prior and the likelihood] also allow us to simulate the observations that the model implies. ...
rtviii's user avatar
  • 111
4 votes
2 answers
292 views

Will synthetic data (e.g. from a generative model) produce valid statistical inference?

Especially with chatGPT now, there is a lot of interest in generative models for healthcare for creating synthetic patient data to amplify patient counts, e.g. for rare diseases. These are then ...
elbord77's user avatar
  • 537
1 vote
1 answer
109 views

How can be decoder of VAE represent probability distribution p(x|z) eventhough we directly get image as output

How can be decoder of VAE represent probability distribution p(x|z) eventhough we directly get image as output. Also if my current understanding is right than we get same image for same value of z ...
Hamit Des's user avatar
1 vote
1 answer
183 views

What kind of Neural Networks are required for Diffusion models?

It appears that regular feed-forward and convolutions are not enough to make diffusion models work (from some personal limited testing, they do not work at all). The typical infrastructure was a U-Net ...
Edv Beq's user avatar
  • 703
0 votes
1 answer
64 views

In factor VAE, do you freeze the discriminator weights during the back propagation step for the FVAE loss?

In factor vae, Disentangling by Factorising, there are two losses that are minimized. One is the VAE loss (eq. 2 in the paper) that includes (1) reconstruction loss, (2) KL divergence and (3) Total ...
S R's user avatar
  • 23
0 votes
0 answers
11 views

Can Denoising Diffusions only learn white noise, and/or no drift?

I understand that it works best for image generation to add white noise and no drift because is simpler. My question is, theoretically speaking, can the noise follow a certain distribution and the ...
Edv Beq's user avatar
  • 703
0 votes
0 answers
38 views

VAE with only forward diffusion enhancement ** experiment **

I wanted to get some opinions with an idea that I have explored for a little bit. This is an experiment and I would like to know if this is mathematically plausible or not. Imagine $\bar{x}$ is the ...
Edv Beq's user avatar
  • 703
0 votes
0 answers
189 views

Should the KL loss term for a VAE be the KL-Loss of a batch's mean mu and log sigma, or is it the mean of the kl loss for each individual input image?

I've been trying to learn about Variational Autoencoders and been looking at the Keras sample implementation (https://github.com/keras-team/keras-io/blob/master/examples/generative/vae.py) I'm ...
user1245262's user avatar
0 votes
0 answers
21 views

What kind of learning is feature generation for score maximization?

What kind of deep learning is the generation of numerical features (Y) from objects (X) used to compute a score (f(.), differentiable) that is to be maximized directly? Basically NN$\theta$(x) = y, f(...
user389279's user avatar
2 votes
1 answer
36 views

Accuracy of probability estimate from generative autoregressive language model

My understanding is that a discriminative classifier such as a CNN that takes an input $x$ and produces a discrete output label $y$ is typically trained to predict the best value of $y$, and would not ...
sunfishstanford's user avatar
4 votes
2 answers
101 views

What are the best resources on image synthesis?

What are some good resources to learn about image synthesis? What are some of the key concepts or architectures to study? I understand image synthesis as generating new images with ML techniques.
Glue's user avatar
  • 463
4 votes
2 answers
194 views

Using Generative Adversarial Networks for joint distribution estimation

I am trying to use GAN model to generate N-dimensional samples with joint probability distribution that looks like some training data. I am having trouble getting the probability distribution of the ...
dvd8719's user avatar
  • 55
1 vote
0 answers
33 views

Expectation of energy-based model

An energy-based model parametrized by $\theta$ is defined as $$ p(x; \theta) \propto \exp(-f(x; \theta)) $$ For my specific case, it is that $f(x; A, y) = - \langle x, Ax \rangle - \langle x, y \...
Vũ Lê Thế Anh's user avatar
0 votes
0 answers
10 views

Network invocations underlying OpenAI execution?

Really a pretty basic question about generative models, but I'm trying to map my (limited) understanding of NNs generally to what's going on when I invoke an OpenAI API: When the OpenAI API docs ...
orome's user avatar
  • 408
3 votes
1 answer
85 views

Blurring of image in generative model using diffusion probabilistic method

In the 2015 paper "Deep Unsupervised Learning using Nonequilibrium Thermodynamics" by Sohl-Dickstein et al. on diffusion for generative models, Figure 1 shows the forward trajectory for a 2-...
sunfishstanford's user avatar
0 votes
1 answer
80 views

VAE active units

According to Burda et al (2015) number of active units is computed as: $ Cov_x(E_{z \sim q_\phi(z|x)}) > \delta $ for some particular delta. In the paper it is set to 0.02 empirically. But this ...
Pavel Podlipensky's user avatar
1 vote
1 answer
32 views

Rewrite linear expression

Is it possible to rewrite $$ \frac{-1}{2}\left(x^T\Gamma^{-1}(\mu_1-\mu_0)+(\mu_1-\mu_0)^T\Gamma^{-1}x\right) $$ as $$ -\theta^Tx $$ where $x,\mu\in\mathbb{R}^d, \Gamma\in\mathbb{R}^{d\times d}$ and $\...
jroc's user avatar
  • 13
0 votes
1 answer
72 views

Normalizing Flows Invertibility

I am currently reading up on RealNVP, which has the following transformations according Lilian Weng: $$ \begin{aligned} \mathbf{y}_{1:d} &= \mathbf{x}_{1:d} \\ \mathbf{y}_{d+1:D} &= \mathbf{x}...
user2793618's user avatar
0 votes
0 answers
24 views

Classification via "Activation Maximization"

I came across the following problem and I'm unsure if a solution exists or if it has been proven that what I'm trying to achieve is impossible. I tried to google for papers on the topic but I haven't ...
Liuka's user avatar
  • 101
8 votes
5 answers
1k views

Why do Denoising Diffusion Probabilistic Models (DDPM) add noise according to $\sigma_t$ during sampling?

Reading about Denoising Diffusion Probabilistic Models (DDPM) the paper (algorithm 2 - sampling) states that the sampling goes according to... $$ x_{t-1} = \frac{1}{\sqrt{\alpha_t}}\left( x_t - \frac{...
Joff's user avatar
  • 872
2 votes
1 answer
779 views

How to generate new data with a VAE?

I have built the following function which takes as input some data and runs a VAE on them: ...
quant's user avatar
  • 511
4 votes
1 answer
863 views

How to rewrite DreamBooth loss in terms of $\epsilon$-prediction?

I'm trying to make the loss used in DreamBooth paper explicit, writing it in terms of the noise, as it is commonly written in the original diffusion article [1], instead of the image reconstruction ...
Ciodar's user avatar
  • 475
3 votes
0 answers
324 views

If neural networks are deterministic once trained how are generative able to generate different responses to inputs?

Once training is finished and a neural network is in inference mode, its outputs will be deterministic, i.e. the weights have been fixed. How is it then that generative models are able to generate non-...
Jose Miguel Cruz y Celis's user avatar
1 vote
1 answer
54 views

Bayes theorem in the context of generative classification models

In the Introduction to Statistical Learning p. 142 in chapter 4.4 on generative models for classification the formula $P(Y = k|X = x) = \frac{π_k \cdot f_k (x)} {\sum_{l=1}^{K}π_l f_l(x)}$ is given to ...
LololoTheGreat's user avatar
10 votes
1 answer
1k views

Purpose of scaling mean by $\sqrt{1 - \beta_t}$ in forward diffusion process

In the forward diffusion process described by Ho, et al. the probability distribution for the next step is: $$q(\mathbf{x}_t|\mathbf{x}_{t-1}) = N(\mathbf{x}_t;\sqrt{1-\beta_t}\mathbf{x}_{t-1},\beta_t\...
Adrian Stoll's user avatar
0 votes
0 answers
44 views

Adjusting weight updates in a generative model

Suppose I am training a generative model G to produce vectors z in R^d, where d is fixed. The objective of G is to produce realistic vectors, which I am calling the "reality objective". ...
postnubilaphoebus's user avatar
2 votes
1 answer
420 views

How is the variance for a diffusion kernel derived for a diffusion model?

So I'm watching this video tutorial from CVPR this year on diffusion models, and I am confused by the variance term in the distribution on the left on the video. I understand that in the forward ...
Cynthia Kim's user avatar
0 votes
1 answer
259 views

Deriving the Reparameterization Trick

I am reading about the reparameterization trick from here. It states $\boldsymbol{\epsilon}\sim p(\boldsymbol{\epsilon})$, $\textbf{z}=g_\theta(\boldsymbol{\epsilon},\textbf{x})$, and $$\mathbb{E}_{p_\...
muser's user avatar
  • 135
3 votes
2 answers
1k views

Can the dimension of the latent space in VAEs, be larger than the dimension of the data?

I am experimenting with VAEs. There, there is a parameter that you need pass when you create the NN, which is the dimension of the latent space. In the typical ...
quant's user avatar
  • 511
1 vote
1 answer
160 views

VAE mean and Standard Deviations are input dependent?

The original presentation of variational autoencoders, VAE assumes the mean $\mu$ and the sd $\sigma$ are functions of the input variable, say $x$. I am studying "Learning Structured Output ...
Toonia's user avatar
  • 13
1 vote
2 answers
209 views

Autoregressive models for modeling conditional probabilities

I'm reading Tomczak's Deep Generative Modeling. When the author discusses auto-regressive models, he mentions that we model the probability distribution $p(\mathbf{x})$ of the data $\mathbf{x}$ as $$p(...
An Ignorant Wanderer's user avatar
1 vote
1 answer
102 views

Understanding conditional probability formulas in the context of class-conditionals in generative models

I am trying to understand the theory behind probabilistic generative models a bit better. If I model the class-conditionals as Gaussians, the formula is this: $$ \frac{1}{2\pi^{\frac{D}{2}}|\Sigma|^...
user3813234's user avatar
0 votes
0 answers
231 views

What's GAN's Input-size Limitations?

I am interested in GAN for generating synthetic data. I am studying the input limitations for GAN starting from which GAN is no longer usable. I have found many applications that use GANs for ...
AbdelKh's user avatar
  • 101
1 vote
0 answers
51 views

How to implement simple VAE with sparse tensor in Tensorflow

thank you for reading. I have been attempting to train a simple VAE on very sparse 2D and 3D data. So far I have been training using dense tensors which - I think - is resulting in horrible training ...
Zephrom's user avatar
  • 11
1 vote
0 answers
114 views

Can you make a simple 2D ring with a GAN?

I am trying to model simple 2d continuous distributions with GANs. Here, I focus on a 2d distribution following a ring structure. The architecture of my networks are: ...
Florian Lalande's user avatar
1 vote
1 answer
53 views

Can posterior become tractable if we know p(x)?

In the VAE framework where x is an input data (a vector) and z is a vector of continuous latent variables, the posterior ...
MerelyLearning's user avatar
2 votes
1 answer
71 views

Parametric vs non-parametric generative models

I have a little perplexity trying to distinguish parametric vs non-parametric generative model. In my understanding, a parametric generative model would try to learn the probability density function ...
James Arten's user avatar
2 votes
1 answer
863 views

What is the exact role of model $p_\theta$ in Diffusion models for the reverse process?

I'm reading this interesting blog post explaining Diffusion probabilistic models and trying to understand the following. In order to compute the reverse process, we need to consider the posterior ...
James Arten's user avatar

1
2 3 4 5 6