reparameterization trick in VAEs, How should we do this?

Question

I'm confused about how does reparameterization trick works. In this article shows it very simple. You learn two vectors $\sigma$ and $\mu$, sample $\epsilon$ from $N(0, 1)$ and then your latent vector $Z$ would be (where $\odot$ is the element-wise product.): $$ Z = \mu + \sigma\odot\epsilon $$ BUT when I look at the TensorFlow tutorial code for VAEs, it is not just a simple $\odot$. The code is this:

  def reparameterize(self, mean, logvar):
    eps = tf.random.normal(shape=mean.shape)
    return eps * tf.exp(logvar * .5) + mean

which is showing this: $$ Z = \mu + \epsilon\times e^{0.5\times\log{}var} $$

These two are not the same and I'm confused,

1. first why does it learn the logarithm of variance (as the name of the variable suggests) instead of learning just variance.
2. second, why it is multiplied by 0.5?
3. and finally, which one is the correct reparameterization trick (if they differ)?

Answer 1

1. We do this simply because the variance of a random variable is constrained to be positive (i.e. $\sigma^2 \in \mathbb{R}^+$) and so if we were to try to learn the variance we would have to constrain somehow the output of a neural network to be positive. A simple way around this is to learn the logarithm instead since $\log(\sigma^2) \in \mathbb{R}$ ensures that $\exp(\log(\sigma^2)) \in \mathbb{R}^+$.

2. The $\sigma$ in the first equation is the standard deviation which as you know is the square root of the variance. Then you can see that the multiplication of 0.5 outside the log equates to raising the variance inside of the log to the power of 0.5:

$$ e^{0.5\log(var)} = e^{\log(var^{0.5})} = var^{0.5} = \sigma $$

3. Given the answer to (2) it should be clear they are the same.