Why does the first forward pass in a neural-network (NN) classification model computes to zero for all classes before final activation? Suppose weights of NN are Gaussian spread initialization then forward pass for all the inputs will evaluate to zero which computes to 0.69 ($-\log_{e}0.5 \approx 0.69 $, since sigmoid (0) = 0.5) average loss at least in the first pass. For three classes it would be $1.0986$ ($-\log_{e}\frac{1}{3}$) and so on. 
I understand when weights are zero-initialized, but why does this happen when weights are Normally distributed to start with?       
 A: When the weights are zero-initialized, it's certain that you get $-\log 0.5$ in the first batch. In normal, this is not guaranteed, but you'll get similar results on average. Because, on average, each input to logistic regression will be $E[w^Tx+b]=E[w^T]x+E[b]=0$, because $E[w]=E[b]=0$. Actually, each input to sigmoid function is going to be normally distributed with mean $0$ and variance some $\sigma^2$, which can be estimated from your initialization variances for each parameter.
For sigmoidal output, we have the following expected value:
$$E\left[\frac{1}{1+e^{-v}}\right]=\int_{-\infty}^\infty \frac{e^{-v^2/2\sigma^2}}{\sqrt{2\pi}\sigma}\frac{1}{1+e^{-v}}dv=\frac{1}{2}$$
This integral result can be verified from wolfram, which is hard to compute, probably via methods using contour integrals; however, very intuitive if you look at the sigmoid's graph. We normally don't have $E[f(X)]=f(E[X])$, but in this case it holds.
What we're finally interested in is the loss expression, i.e. $E\left[\log\left(1+e^{-v}\right)\right]$, which'll be harder to compute and not available as a theoretical result in wolfram alpha integrator, at least in free version. But, now, it'll give different values according to your initialization variance.
Standardizing your inputs, and using small variances like $1/n$ where $n$ (similar to Xavier init.) is the number of neurons will give you approximately $-\log 0.5$ as loss. 
