Variational Autoencoder, understanding this diagram I'm not an ML scientist, but I'm trying to understand how variational autoencoder works.
I'll take as reference the following diagram, which it couldn't be used for backpropagation as includes a sampling process but it captures anyway what I don't understand. The diagram is taken from this link.

I'm specifically going to focus on the encoder part. My understanding is that $x_1,\ldots, x_6$ are real values (the features) and in the second layer each of $a_1, \ldots\ a_4$ is another real value.
No we have these functions $\mu_1,\mu_2,\sigma_1,\sigma_2$, from that diagram again it seems that both $\mu_1$ and $\mu_2$ compute the mean of the vector $a = (a_1,\ldots,a_4)$ but if this is the case then $\mu_1(a) = \mu_2(a)$ and I don't see the point of this, I'd make a similar observation for the $\sigma_1, \sigma_2$ functions.
The question is, w.r.t. that diagram, how exactly are $\mu_1$ and $\mu_2$ computed?
 A: The point of a variational autoencoder is to have an encoder that produces a probability distribution for a given input. In this model, the latent probability distribution is 2 independent normals, equivalently a bivariate normal distribution with mean vector $\begin{bmatrix}\mu_1 \\ \mu_2 \end{bmatrix}$ and covariance matrix $\begin{bmatrix} \sigma_1^2 & 0 \\ 0 & \sigma_2^2 \end{bmatrix}$. Each input is mapped to its own probability distribution. Then you sample from that distribution, and the decoder reconstructs the input given that random draw from the distribution.
Importantly, $\mu$ and $\sigma$ are not parameters of the network. They are the outputs of the encoder. The way that the model finds good values of $\mu$ and $\sigma$ is by updating the parameters (weights and biases) of the network.
With this in mind, it's important to recognize that $\mu_i$ doesn't compute the mean of $a$; it's an estimate of the mean parameter $\mu_i$ for that observation. Likewise, $\sigma$ is an estimate of the covariance matrix for the latent probability distribution. When your model learns a disentangled latent representation, each component $i$ corresponds to a different feature of that latent representation, so they will not be equal in general.
More details and general information about VAEs are available in this thread: What are variational autoencoders and to what learning tasks are they used?
