# Fast dropout: How to compute the mean and variance of the approximating Gaussian?

In the Fast Dropout article by Wang and Manning, they talk about approximating the input to a hidden layer by a Gaussian with the same mean and variance as the inputs to the layer (see page 5). But they never discuss how the mean or the variance would be computed. Do they select a minibatch and take statistics over the various elements of the minibatch each time we move to the next layer?

For example, suppose we have a minibatch $(\mathbf{x}_i, y_i)_{1 \leq i \leq n}$. Suppose we've just moved to layer $5$, and $f(\mathbf{x})$ represents the input to layer $5$ given input $\mathbf{x}$ to layer $0$. Then in order to find the appropriate Gaussian, we look at the mean and the variance of $\{f(\mathbf{x}_i)\}_{1 \leq i\leq n}$ in order to find the right Gaussian? (Sort of like we do in batch normalization?)

So, instead of outputting "values" of transformed samples $f(\mathbf{x})$, units output means and variances thereof: $\mathbb{E}[f(\mathbf{x})]$ and $\mathrm{var}[f(\mathbf{x})]$. You can use these in the next layer as the means and variances.
In the first layer, you consider the values of $\mathbf{x}$ to represent the individual means and variances are set to 0.