0
$\begingroup$

If you plot the forward activations over time for each layer, are there any patterns that could explain why the network isn't converging, slow learning, etc?

$\endgroup$
1
$\begingroup$

This depends on the type of the network.

Activations of convolutional networks are much easier to interpret because they generally preserve spatial structure. There are many works on visualization of feature maps in convnets, this page is a good starter: http://cs231n.github.io/understanding-cnn/.

Interpretation of fully connected networks is not that much established. One of the things you can see are "dead neurons", whose activations are always zero or constant.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.