What is the exact role of model $p_\theta$ in Diffusion models for the reverse process?

Question

I'm reading this interesting blog post explaining Diffusion probabilistic models and trying to understand the following.

In order to compute the reverse process, we need to consider the posterior distribution $q(\textbf{x}_{t-1} | \textbf{x}_t)$ which is said to be intractable ' because it needs to use the entire dataset and therefore we need to learn a model $p_\theta$ to approximate these conditional probabilities in order to run the reverse diffusion process'.

If we use Bayes theorem we have

$$q(\textbf{x}_{t-1} | \textbf{x}_t) = \frac{q(\textbf{x}_t |\textbf{x}_{t-1})q(\textbf{x}_{t-1})}{q(\textbf{x}_t)}$$

I understand that indeed we don't have any prior knowledge of $q(\textbf{x}_{t-1})$ or $q(\textbf{x}_t)$ since this would mean already having the distribution we are trying to estimate. Is this correct?

The above posterior becomes tractable when conditioned on $\textbf{x}_0$ and we obtain

$$q(\textbf{x}_{t-1} | \textbf{x}_t , \textbf{x}_0) = \mathcal{N}(\tilde{\bf{\mu}}(\textbf{x}_t , \textbf{x}_0) \, , \, \tilde{\beta}_t \textbf{I})$$

So apparently we obtain a posterior that can be calculated in closed form when we condition on the original data $\textbf{x}_0$. At this point, I don't understand the role of the model $p_\theta$ : why do we need to tune the parameters of a model if we can already obtain our posterior?

Answer 1

I understand that indeed we don't have any prior knowledge of $q(x_{t−1})$ or $q(x_t)$ since this would mean already having the distribution we are trying to estimate. Is this correct?

Yes, I think that $q(x_{t-1})$ could only be estimated with an integration involving the whole dataset (which is intractable), as stated in this blog.

So apparently we obtain a posterior that can be calculated in closed form when we condition on the original data $x_0$. At this point, I don't understand the role of the model $p_\theta$: why do we need to tune the parameters of a model if we can already obtain our posterior?

Knowing only the distribution $q(x_{t-1}|x_t,x_0)$ could not allow sampling, i.e. to generate images from noise, since $x_0$ is a sample from the training dataset.

That's why we want an estimate $p_\theta(x_{t-1}|x_t)$ of $q(x_{t-1}|x_t,x_0)$, which allows to pass some noise $x_T \sim \mathcal{N}(0,\mathbb{I})$ to the model and turn it into an image, with a sufficient number of denoising steps.