0
$\begingroup$

I'm looking through the paper on variationl inference in normalizing flow and have difficulties with understanding some ideas.

I know there are latent variables $\mathbf{z}_i$ and observed variables $\mathbf{x}$, the final distribution corresponds to $\mathbf{z}_K$, but I can't figure out what observed variables $\mathbf{x}$ are.

Is this initial distribution, for example, Gaussian one? Then why in the paper do the authors call q($\mathbf{z}_0$) initial distribution (Page 3, after expression 7)? I thought q($\mathbf{z}_0$) is the output of the first hidden layer of the network.

The $\mathbf{x}$ is not the weights or hyperparameters of the transformation layers because there are different variables for them, namely $\phi$ and $\theta$ (See page 6, Algorithm 1). Then what is the relation between $\mathbf{x}$ and $\mathbf{z}_0$? enter image description here

$\endgroup$

1 Answer 1

0
$\begingroup$

Here is the thing, I think the authors use the notation in eqn.4 in reverse order i.e, $z_L \rightarrow z_{L-1} ... \rightarrow z_1 \rightarrow x$. In this notation $z_L$ has a normal distribution $N(0,I)$ and x is the data distribution whose likelihood is supposed to be maximized. Section 3.1 provides background on the theory of transforming random variables via invertible functions. Now, $z_0$ in this section is some random variable you begin with and then by applying transformations $f_1, f_2, .... f_K$, you get $z_K$ which is another random variable having density given by $q(z_K)$. x is not the hyperparameters of the model, it is the data that you use to learn the model. I believe the Finally, to make the relationship between $z_0$ and $x$ explicit, one can write $z_{0} = f_{1}^{-1}(f_{2}^{-1}(...(f_{K}^{-1}(z_K))..))$ and then $x = f_{0}^{-1}(z_0)$.

$\endgroup$
1
  • $\begingroup$ Thanks, but the paragraph 4.2 says that $q_0$ is Gaussian and it is initial density $\endgroup$ Jul 1, 2020 at 20:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.