Is batch normalization useful outside of convolutional networks? I read that batch normalization was widely used in ConvNets. Why is that? Is it also useful in other types of networks? If not, why not?
Thanks
 A: Re: "I read that batch normalization was widely used in ConvNets. Why is that?"
Quoted from this paper,  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift by Sergey Ioffe, Christian Szegedy 
"We refer to the change in the distributions of internal
nodes of a deep network, in the course of training, as Internal
Covariate Shift. Eliminating it offers a promise of
faster training. We propose a new mechanism, which we
call Batch Normalization, that takes a step towards reducing
internal covariate shift, and in doing so dramatically
accelerates the training of deep neural nets. It accomplishes
this via a normalization step that fixes the
means and variances of layer inputs. Batch Normalization
also has a beneficial effect on the gradient flow through
the network, by reducing the dependence of gradients
on the scale of the parameters or of their initial values.
This allows us to use much higher learning rates without
the risk of divergence. Furthermore, batch normalization
regularizes the model and reduces the need for
Dropout (Srivastava et al., 2014). Finally, Batch Normalization
makes it possible to use saturating nonlinearities
by preventing the network from getting stuck in the saturated
modes.
...
Merely adding Batch Normalization to a state-of-theart
image classification model yields a substantial speedup
in training. By further increasing the learning rates, removing
Dropout, and applying other modifications afforded
by Batch Normalization, we reach the previous
state of the art with only a small fraction of training steps
– and then beat the state of the art in single-network image
classification. Furthermore, by combining multiple models
trained with Batch Normalization, we perform better
than the best known system on ImageNet, by a significant
margin."
