# Why do we say that we're "predicting" the mean/noise in diffusion models?

In DDPM, $${\tilde\mu}_t$$ is the mean of the conditional distribution $$q(x_{t-1}|x_t,x_0)$$ while the neural network $$\mu_\theta$$ is modeling a different conditional distribution $$p_\theta(x_{t-1}|x_t)$$. It just happens that the objective function (ELBO) includes the $$l_2$$ loss $$||{\tilde\mu}_t(x_t,x_0)-\mu_\theta(x_t,t)||^2$$, and the author claimed "the most straightforward parameterization of $$\mu_\theta$$ is a model that predicts $${\tilde\mu}_t$$". I wonder why we can say that, given that $$\mu_\theta$$ and $${\tilde\mu}_t$$ do not even concern the same distribution. I understand that mathematically, the loss would be minimized if $$\mu_\theta$$ is close to $${\tilde\mu}_t$$. But I'm confused by the fact that the neural net is supposed to predict something that it does not model.

The question extends to the noise. By reparameterization, it turns out that $$\epsilon_\theta$$ is "intended to predict $$\epsilon$$ from $$x_t$$ to minimize the loss. However, if we indeed are predicting the total noise, then during sampling, why don't we just remove the total noise from $$x_T$$ in one step?

Your mentioned $$l_2$$ loss comes from the $$\mathbb{E}_q[\sum_{t>1}L_{t-1}]$$ term of the ELBO, where $$L_{t-1}=D_{KL}(q(\mathbf{x}_{t−1}|\mathbf{x}_t, \mathbf{x}_0) || p_\mathbf{θ}(\mathbf{x}_{t−1}|\mathbf{x}_t))$$. So the objective is to find the parameters of the neural network to minimize expectation of the sum of these KL divergences wrt the forward diffusion process's variational distribution $$q(\mathbf{x}_{1:T}|\mathbf{x}_0)$$, and since both the conditional distributions $$p,q$$ here are Gaussians so you arrive at a closed form of each $$L_{t-1}$$ term as another expectation of your $$l_2$$ loss wrt the same variational distribution. And indeed the paper's suggestion that $$\mathbf{\mu_\theta}$$ predicts $$\mathbf{\tilde\mu}_t$$ may not be optimal but most straightforward. Though you correctly claimed $$\mathbf{\mu_\theta}$$ and $$\mathbf{\tilde\mu}_t$$ as posterior means do not concern the same (Gaussian) distribution, but according to the form of the above $$l_2$$ loss, so long as they're close, the ELBO optimization as inference of the diffusion model's hierarchical latent variables can succeed.