Skip to main content

Questions tagged [diffusion]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
1 answer
16 views

simplified form for CDF of inverse Gaussian function, as function of the drift parameter

I am thinking about modelling the probability of an event as a waiting time for a Brownian motion with drift to pass a certain barrier. Then consider what the expression for that probability looks ...
Sextus Empiricus's user avatar
0 votes
0 answers
20 views

Why reverse diffusion process is not a gaussian distribution?

The forward diffusion process, which goes from x_t to x_{t+1} is Gaussian, which is very reasonable as we go the next state by adding random gaussian noise. However, I do not understand why the ...
levitatmas's user avatar
0 votes
1 answer
15 views

Reverse distribution in DDPMs

In Denoising Diffusion Probabilistic Models, we want to reverse a forward process where we blend gaussian noise into a data point $\mathbf{x}_0$ over $T$ steps. The result of each forward step applied ...
CoconutFred's user avatar
0 votes
0 answers
12 views

Parameterization of the reverse process in discrete state space paper

In section 3.3. of the paper Structured Denoising Diffusion Models in Discrete State-Spaces the author defines the following parameterization used in both the reverse and sampling process. $$p_\theta(...
stacker87's user avatar
0 votes
0 answers
8 views

How to interpret Diffusion Maps for the iris dataset?

This might be a poor exercise but I'm trying to understand the methods of paper and if it makes sense to adapt my linear-based workflow with PCA to non-linear manifold methods; thought trying out ...
O.rka's user avatar
  • 1,462
0 votes
0 answers
85 views

Why don’t diffusion models suffer posterior collapse?

In VAEs, posterior collapse occurs when the approximated posterior $q_\theta(z|x)$ becomes the standard Gaussian prior $p(z)$ after training (Lucas et al. 2019). The forward process of diffusion ...
entropy07's user avatar
0 votes
0 answers
9 views

Estimating jump diffusion parameters from first passage time data?

I know that there is a literature on approximating the first passage time distribution of jump diffusion processes. I know there is also a literature on estimating parameters of jump diffusion ...
ThinkConnect's user avatar
2 votes
1 answer
55 views

How to calculate the score of a new datapoint by a score based diffusion model(song & ermon, 2019)?

I have a pretrained score based diffusion model trained on 64X64 images. Now I want to calculate the score of a new image(of same dimension) through this pre-trained neural network. The score network ...
rajoy99's user avatar
  • 21
0 votes
1 answer
115 views

Understanding the function of attention layers in a convolutional neural network (U-Net in a diffusion model)

I am trying to understand the neural network architecture used by Ho et al. in "Denoising Diffusion Probabilistic Models" (paper, source code). They include self-attention layers in the ...
Rational Function's user avatar
1 vote
1 answer
110 views

Should the loss terms in diffusion model training be weighted by the diffusion schedule?

I'm following Understanding Diffusion Models to write my own diffusion model training code. I'm working based off the loss showed in equation 58 In equation 99, a useful form of the individual terms ...
Alexander Soare's user avatar
0 votes
0 answers
20 views

Is the score function in Langevin dynamics related to the informant (score)?

In Langevin dynamics and diffusion models we see the score $\nabla_{\mathbf{x}}\log p(\mathbf{x})$. Here the notation seems to suggest we're taking partial derivatives w.r.t the free $\mathbf{x}$ ...
flinty's user avatar
  • 151
2 votes
1 answer
564 views

How does the QR-code monster model for SD controlNet work?

Background Recent trends show cool QR-codes made by the QR-code monster model used in Stable-Diffusion (SD) with ControlNet. Those are also used to create images containing hidden text/hidden images (...
snatchysquid's user avatar
1 vote
1 answer
296 views

What kind of Neural Networks are required for Diffusion models?

It appears that regular feed-forward and convolutions are not enough to make diffusion models work (from some personal limited testing, they do not work at all). The typical infrastructure was a U-Net ...
Edv Beq's user avatar
  • 713
0 votes
0 answers
15 views

Can Denoising Diffusions only learn white noise, and/or no drift?

I understand that it works best for image generation to add white noise and no drift because is simpler. My question is, theoretically speaking, can the noise follow a certain distribution and the ...
Edv Beq's user avatar
  • 713
0 votes
0 answers
76 views

What are the similarities and the differences between MaskGit and Diffusion models

maskgit and diffusion - can you explain the similarities and differences between these two types of models?
ihadanny's user avatar
  • 3,340
4 votes
1 answer
332 views

Reparameterization of Poisson Distribution

In deep learning, especially generative models, sometimes we need to add some random noise to the input of model. To make the sampling of random noise learnable (or differentiable), we need to ...
Lorin60's user avatar
  • 91
0 votes
0 answers
162 views

Confusion with the "lower bound"-term in diffusion models

I am trying to understand the maths of diffusion models following this video explanation on youtube and this blog post. Here is what how I understood it so far: The overall goal is, that we want to ...
mayool's user avatar
  • 1
0 votes
0 answers
41 views

VAE with only forward diffusion enhancement ** experiment **

I wanted to get some opinions with an idea that I have explored for a little bit. This is an experiment and I would like to know if this is mathematically plausible or not. Imagine $\bar{x}$ is the ...
Edv Beq's user avatar
  • 713
3 votes
1 answer
101 views

Blurring of image in generative model using diffusion probabilistic method

In the 2015 paper "Deep Unsupervised Learning using Nonequilibrium Thermodynamics" by Sohl-Dickstein et al. on diffusion for generative models, Figure 1 shows the forward trajectory for a 2-...
sunfishstanford's user avatar
0 votes
1 answer
132 views

Parameterizing a Gaussian distribution

I am reading this blog post where the author talks about diffusion models. Let's keep diffusion out of the conversation for now. The author showcased that we can parameterize a Gaussian distribution ...
enterML's user avatar
  • 378
3 votes
1 answer
724 views

Justification of the fixed variational distribution in diffusion models

Diffusion models can be regarded as latent variable models (Ho et al., 2020; Section 2), with the latents being an hierarchical chain of random variables $z_T → \dots → z_t → z_{t-1} → \dots → z_1$ (...
Dan Oneață's user avatar
4 votes
1 answer
931 views

How to rewrite DreamBooth loss in terms of $\epsilon$-prediction?

I'm trying to make the loss used in DreamBooth paper explicit, writing it in terms of the noise, as it is commonly written in the original diffusion article [1], instead of the image reconstruction ...
Ciodar's user avatar
  • 485
12 votes
1 answer
1k views

Purpose of scaling mean by $\sqrt{1 - \beta_t}$ in forward diffusion process

In the forward diffusion process described by Ho, et al. the probability distribution for the next step is: $$q(\mathbf{x}_t|\mathbf{x}_{t-1}) = N(\mathbf{x}_t;\sqrt{1-\beta_t}\mathbf{x}_{t-1},\beta_t\...
Adrian Stoll's user avatar
-1 votes
1 answer
536 views

Why is the square root in the forward process of the Diffusion Model?

I am trying to make sense of the diffusion model (one of the videos I watched https://www.youtube.com/watch?v=HoKDTa5jHvg&t=706s). I came across this formula in the model: This picture clearly ...
yts61's user avatar
  • 109
5 votes
2 answers
2k views

Diffusion Models - modeling noise?

In this post about diffusion models, IIUC, we want to use a neural network to approximate the mean of the reverse diffusion: $p_\theta(\mathbf{x}_{t-1} \vert \mathbf{x}_t) = \mathcal{N}(\mathbf{x}_{t-...
Victor M's user avatar
  • 319
3 votes
2 answers
630 views

How is the variance for a diffusion kernel derived for a diffusion model?

So I'm watching this video tutorial from CVPR this year on diffusion models, and I am confused by the variance term in the distribution on the left on the video. I understand that in the forward ...
Cynthia Kim's user avatar
2 votes
1 answer
157 views

ELOB (evidence lower bound) for diffusion

I am trying to understand the loss definition for diffusion, but lots of questions already arise during the first step of the derivation. (I have nearly zero statistic knowledge, but I am good at ...
Raven Cheuk's user avatar
3 votes
1 answer
1k views

What is the exact role of model $p_\theta$ in Diffusion models for the reverse process?

I'm reading this interesting blog post explaining Diffusion probabilistic models and trying to understand the following. In order to compute the reverse process, we need to consider the posterior ...
James Arten's user avatar
1 vote
0 answers
37 views

Understanding why volatility in diffusion process $(X_t)_{t \in[0,T]}$ is identifiable/known for continuous observations, but the drift is not?

Why is it that when dealing with continuous time observations of a diffusion process $(X_t)_{t \in[0,T]}$, we say that the volatility $\sigma^2$ is "perfectly identifiable" and just usually ...
Jacobiman's user avatar
  • 147
2 votes
1 answer
586 views

Sampling distribution of GBM Maximum-Likelihood estimator

Given the geometric Brownian diffusion $$ X_t = \mu X_t \, dt + \sigma X_t \, d W_t$$ I learnt that its maximum likelihood estimators are the following as this web article suggests $$\hat \mu = \frac{\...
student's user avatar
  • 261
0 votes
0 answers
64 views

Jump diffusion -advantages

What would people say is the advantage of using a Merton jump-diffusion model, in terms of what it models and it's key characteristics/ features?
Carly Johnson's user avatar
0 votes
0 answers
89 views

How are differential equations and stochastic differential equations different?

In the simplest terms, how are differential equations and stochastic differential equations different? As far as I can tell, SDEs are PDEs or ODEs, where the derivative of some function wrt itself is ...
jbuddy_13's user avatar
  • 3,298
3 votes
0 answers
331 views

Carré du champ operator is a quadratic variation

Let $X_t$ be a real valued Markov process (starting at $x$) with generator $L$. Let $\Gamma(f)$ denote Carré du champ operator i.e. $L(f^2) - 2f \cdot L (f)$. As far as I know under suitable ...
marcusy's user avatar
  • 43
1 vote
0 answers
58 views

Fit drift diffusion model with trial-type dependent input strength

I want to fit a drift diffusion model to a task which involves multiple decisions (n=400) between two different valuable choice options . I do understand how I would do that in general, also with the ...
Laurie's user avatar
  • 319
2 votes
0 answers
42 views

What is the likelihood function of the starting time of diffusion?

I need to find the likelihood that a set of molecules was instantaneously released at time $t_0$, say $t_0=0$. Toy System Example: Let $N$ be the set of molecules released from a specific point in a ...
nashynash's user avatar
2 votes
0 answers
37 views

How to apply the diffusion maps when the matrix is PSD but not positivity preserving?

In order to apply the diffusion maps in a matrix $C\in\mathbb R^{n\times n}$ , that matrix must obey some restrictions, C is symmetric: $C_{ij} = C_{ji}$, C is positivity preserving (PP): $\forall ...
daemon's user avatar
  • 21
2 votes
0 answers
235 views

Diffusion tensor as a covariance matrix

TLDR: In nuclear magnetic resonance (NMR), to study molecular diffusion we assume that molecules displace in 3D space according to a trivariate gaussian distribution. The variables are then the ...
user241848's user avatar
2 votes
1 answer
384 views

What is the distribution of the peak time of the first hitting time process

I need to find the distribution of the random variable $T_{peak}$ where $T_{peak}$ represents the peak time of the first hitting time process. Detailed Explanation of the System: There are $N^{Tx}$ ...
fermat4214's user avatar
2 votes
0 answers
80 views

Stochastic Differential equation: CAPM

Let $R = (R_1, \dots , R_M)'$ denote a vector of excess returns of $M$ assets observed at $n$ time points, $0 < t_1 < t_2 < \cdots < t_n < T$, within a time span $T > 0$. We wish ...
bardwell's user avatar
3 votes
0 answers
29 views

Increase the number of samples when the PDF is invariant

Background: $$\frac{\partial C}{\partial t} = D \frac{\partial^2 C}{\partial x^2}$$ is given by Fick's second law, in which $D$ is the diffusion coefficient. The solution to this equation (given the ...
meTchaikovsky's user avatar
0 votes
1 answer
73 views

Why is the sample variance equal to the average sample size for random processes?

I am learning auto-correlation function for fluorescence correlation spectroscopy (FCS) on fcsXpert.com. The web page says: For random processes such as diffusion, the average of the square of the ...
FAN FAN's user avatar
0 votes
1 answer
104 views

Distribution of the step size of diffusion in 3-dimensional space

I need to find the distribution of the random variable $$Y=\sqrt{X_1^2+X_2^2+X_3^2}$$ where $X_i\sim{\cal{N}}(0,\sigma^2)$ and $\sigma^2$ is related to diffusion coefficient. All $X_i$s are ...
fermat4214's user avatar
2 votes
0 answers
377 views

How to simulate anomalous diffusion of a 1D point like particle?

I want to simulate 3 types of diffusion processes: normal diffusion $[\langle x^2(t)\rangle \propto t ]$. subdiffusion $[\langle x^2(t)\rangle \propto t^\alpha ; \alpha<1 ]$ superdiffusion $[\...
0x90's user avatar
  • 739
1 vote
0 answers
89 views

Computing properties of non-uniform random walk/diffusion

I have a lot of numerical data which I'm looking to characterise as a (possibly continuous) random walk with variable (in space) step size, for example, along $x$ between $-1$ and $1$ with a step size ...
Sean D's user avatar
  • 11
1 vote
0 answers
273 views

2 Dimensional Random Walk Simulation

I am trying to simulate random diffusion of particles using a random walk diffusion model. I have used probabilities of movement of particles in a 2D area, to be 1/4 in all 4 directions. The confusion ...
Rizwan Ali's user avatar
1 vote
2 answers
974 views

confusion about root mean squared distance in 1 dimensional random walk

I was just introduced to the concept of a random walk while reading the Feynman lectures on physics, Volume 1. There was something in the explanation there that confused me, so I tried looking online ...
spacediver's user avatar
9 votes
1 answer
584 views

Theoretical link between the graph diffusion/heat kernel and spectral clustering

The graph diffusion kernel of a Graph is the exponential of its Laplacian $\exp(-\beta L)$ (or a similar expression depending on how you define the kernel). If you have labels on some vertices, you ...
highBandWidth's user avatar
3 votes
0 answers
583 views

Diffusion coefficient from double-normal probability density function

The spread of individuals of species is often described by so-called dispersal kernels. The main parameter of spread is then often the variance defined as the average squared movement distance of a ...
Johannes's user avatar
  • 155
16 votes
0 answers
412 views

What is tantile regression?

My question follows on this discussion of medials and tantiles vs medians and quantiles from earlier this year: When would we use tantiles and the medial, rather than quantiles and the median? As ...
user78229's user avatar
  • 10.8k
5 votes
2 answers
3k views

Probability distribution of the magnitude of a circular bivariate random variable?

I'm very new to this topic. I have a distribution similar to the picture below but with the center at zero. As I said, I'm very new to this, but if I understand correctly, if there was no hole in ...
Maria's user avatar
  • 53