# Normalizing flow training

I've been learning about normalizing flow. This is my understanding, and please correct me whenever I am wrong.

There are $$\{y_1,y_2,...,y_n\}$$ samples from an unknown distribution $$p_y(y)$$ that we wish to learn. We can consider a distribution that is well-known such as Gaussian with density $$p_y(x)$$. Then, the goal of normalizing flow is to find a $$f_{\theta}$$ with parameters $$\theta$$ such that $$Y=f_{\theta}(X)$$. To find $$p_y(y)$$ or $$\log (p_y(y))$$ we need to employ the change of variable formula, namely: $$\log(p_y(y))=\log(p_x(f_{\theta}^{-1}(y)))-\log\left|\dfrac{dg}{dx} \right |=\log(p_x(x))-\log\left|\dfrac{df_{\theta}}{dx} \right |$$ I know that maximum likelihood estimation is used to find $$\theta$$. However, I am not sure exactly how that works. I'd appreciate if someone could show me some details on that. The examples I've seen online usually just use a python library and details are not included.

*Edit: my question was: how are the data points $$\{y_1,y_2,...,y_n\}$$ exactly used to the gradient ascent on the likelihood function?

• Which details are confusing to you? We can give more helpful answers if you tell us where particularly you're stuck. Apr 10, 2021 at 22:31
• I don't know how the maximum likelihood estimation is executed. In particular, I would not know how use $\{y_1,y_2,...,y_n\}$ and $\{x_1,x_2,...,x_n\}$ when doing the estimation. Apr 10, 2021 at 22:42
• Do you understand how MLE works when there isn't a normalizing flow? Apr 10, 2021 at 22:46
• yes, I am familiar with MLE, just not with normalizing flow. Apr 10, 2021 at 22:53

Overview.
You're familiar with MLE, which is a good starting point. We have a parametric model whose parameters $$\theta$$ we seek to optimize, in order to maximize the likelihood of our model $$L(\theta \mid \{y_1,y_2,...,y_n\}) = p_Y(\{y_1,y_2,...,y_n\} \mid \theta)$$. Gradient-based methods are one way to do the optimization.

It's no different with normalizing flows: we're trying to maximize the likelihood. The transformation $$f$$ has some parameters $$\theta$$ that we seek to optimize, possibly with gradient ascent. (The prior distribution on $$x$$, $$p_X$$, may have its own parameters $$\phi$$ to learn. Or we could just declare it to be a standard Gaussian with the same dimension as $$Y$$ or something. This means that only $$f$$ has learnable parameters.) These parameters are just like any other.

About $$f$$.
The transformation $$f$$ must have three important properties that we will use in a moment. It is deterministic; it is invertible; and it has an easily computable, easily differentiable determinant of the Jacobian. Some example functions satisfying these requirements are here.

Computing the likelihood.
Mechanically, how would you compute $$p_Y(y_i \mid \theta)$$? Remember that $$p_Y(y_i \mid \theta) = p_X\left(f^{-1}_\theta(y_i)\right) \cdot \left|\det \left(\frac{\mathrm{d}f^{-1}_\theta}{\mathrm{d}x}\right)\right|$$. (Your question was missing the $$\det$$.)

1. Apply the inverse transform of your observed variable $$y_i$$ to compute the latent variable $$x_i = f^{-1}_\theta(y_i)$$.
2. Compute the probability density $$p_X(x_i)$$.
3. Compute the $$\left|\det \left(\frac{\mathrm{d}f^{-1}_\theta}{\mathrm{d}x}\right)\right|$$ term. This is easy because of the functional form we chose.
4. Multiply these terms to get the likelihood.

Learning $$\theta$$.
To actually learn $$\theta$$ with gradient descent requires finding and following the gradient. I'll suggest a way assuming you're working in a computing toolkit that supports reverse-mode automatic differentiation (back-propagation). There are other options as well; heck, you could define the gradients by hand if you hate yourself want to.

1. Backpropagate from the negative log of the likelihood to compute gradients with respect to $$\theta$$.
2. Adjust $$\theta$$ by taking a step in the direction down the gradient.
• So, in terms of computation, there's never a need to sample a Gaussian (assuming we are using Gaussian), namely get $\{x_1,x_2,...,x_n\}$ directly from the Gaussian? Also, thanks for your answer! it's very helpful! Apr 11, 2021 at 0:29
• Right—you never need to sample from $p_X$ if it's simple to evaluate its pdf. If the distribution is more complicated, perhaps you'll need a Monte Carlo estimate of the pdf. Apr 11, 2021 at 2:31