# Generating probability distribution parameters using a neural network

I am new to neural network therefore, my question might be super basic. I am reading this article:

https://arxiv.org/pdf/1609.09869.pdf

In this article they've made a deep Markov Model in which they use recurrent neural network to generate the probability distribution parameters (mu and sigma). That is they have a string of observables (X) and which they feed to a bi-directional neural network and they use these networks to produce mu and sigma which in turn gives them latent variable (z). I've attached the image as well. My question is how can you train your network to give distribution parameters? I know how a neural network works generally. So, for each label we have a corresponding input and with that input we get let's say a probability of whether its a cat or a dog in the final softmax layer. But how can we get mu and sigma for distribution parameters?

This is given in the paper

\begin{align} \mu_r &= W_{\mu_r}^\text{right} h_t ^\text{right} + b _{\mu_r}^\text{right} \\ \sigma_r^2 &= \operatorname{softplus}(W_{\sigma_r}^\text{right} h_t ^\text{right} + b _{\sigma_r}^\text{right}) \\ \mu_l &= W_{\mu_l}^\text{left} h_t ^\text{left} + b _{\mu_l}^\text{left} \\ \sigma_l^2 &= \operatorname{softplus}(W_{\sigma_l}^\text{left} h_t ^\text{left} + b _{\sigma_l}^\text{left}) \\ \mu_t &= \frac{\mu_r\sigma^2_l + \mu_l\sigma^2_r }{\sigma^2_l + \sigma^2_r} \\ \sigma^2_t &= \frac{\sigma^2_l \sigma^2_r }{\sigma^2_l + \sigma^2_r} \\ \end{align}

They are calculated using dense layers, as a function of the outputs of previous layers. What they are assuming in here is that there the parameters are some complicated, possibly non-linear, function of the inputs

$$y_t \sim \mathcal{N}(\mu(x), \sigma(x))$$

You can notice that the last two lines are the update rule for Gaussian distribution given the conjugate priors.

• but how do we use these to get latent variables? Jul 2, 2019 at 10:57
• @Peter get what exactly? The above formulas show how to get them, to estimate the parameters you maximize the likelihood, as described in the paper. The estimated parameters are the latent variables.
– Tim
Jul 2, 2019 at 11:04
• I think I get it now. Thank you @Tim Jul 2, 2019 at 11:07