I am trying to build a Variational Autoencoder. I was looking at various codes online and found most of them in some way or another copy Francois Chollet (Google researchers) code.
Now my main question with this code is this part:
First, here's our encoder network, mapping inputs to our latent distribution parameters:
x = Input(batch_shape=(batch_size, original_dim)) h = Dense(intermediate_dim, activation='relu')(x) z_mean = Dense(latent_dim)(h) z_log_sigma = Dense(latent_dim)(h)
We can use these parameters to sample new similar points from the latent space:
def sampling(args): z_mean, z_log_sigma = args epsilon = K.random_normal(shape=(batch_size, latent_dim), mean=0., std=epsilon_std) return z_mean + K.exp(z_log_sigma) * epsilon # note that "output_shape" isn't necessary with the TensorFlow backend # so you could write `Lambda(sampling)([z_mean, z_log_sigma])` z = Lambda(sampling, output_shape=(latent_dim,))([z_mean, z_log_sigma])
As you can clearly see the $log(\sigma) = \mu$. Where did this assumption come from? How is it possible that we generate a random normal distribution with mean $\mu$ and standard deviation $\sigma$ like this?