If you plot the forward activations over time for each layer, are there any patterns that could explain why the network isn't converging, slow learning, etc?
This depends on the type of the network.
Activations of convolutional networks are much easier to interpret because they generally preserve spatial structure. There are many works on visualization of feature maps in convnets, this page is a good starter: http://cs231n.github.io/understanding-cnn/.
Interpretation of fully connected networks is not that much established. One of the things you can see are "dead neurons", whose activations are always zero or constant.