Sigmoid functions that are symmetric about the origin have been recommended in neural nets literature (for instance, Efficient Backprop by LeCun et al.). Hence, a tanh activation function with a range between -1 and 1 is preferred over a logistic sigmoid with a range between 0 and 1. To save the logistic sigmoid, why are its outputs not normalized (zero mean and unit variance)?
For all real values v, sigmoid(v) is always ∈ (0, 1), and probabilities of events are also always ∈ (0, 1). Thus, the sigmoid function used in the output layer is good for modeling probability values directly, and for this purpose (modeling probabilities) there's no need to normalize the output values.
If adding an extra layer before the sigmoid output layer only for shifting and scaling output values, then this does not improve performance but only adds computational cost. Since theoretically, linear operation, already reflected in weights of weight matrices, does not enhance model complexity.