In most neural networks that I've seen, especially CNNs, a commonality has been the lack of batch normalization just before the last fully connected layer. So usually there's a final pooling layer, which immediately connects to a fully connected layer, and then to an output layer of categories or regression. I can't find it now but, I remember seeing a vague reference on this that concluded batch normalization before the last FC layer didn't make much of a difference. If this is true, why is this?
In practice, it seems like the last FC layer tends to have around 10% of its neurons dead for any given input (although, I haven't measured neuron contiguity). This proportion tends to grow considerably when you increase the FC layer, especially when starting from previously pre-trained models.