Feed Forward Neural Network - How to Visualize the Weight Matrix?

Question

I'm a bit confused about the visualization of the weights of a feed forward neural network as provided in this example from scikit-learn. The network has an architecture of [784 x 50 x 10] (MNIST dataset), so n_hidden = 50 (there are 50 hidden units). Each of the hidden units is a column of the weight matrix W_1 (with dimension 784x50), right? From this, I thought, we could create 50 images, each of size 28x28 (=784). However, there are only 16 images shown? What is my mistake here?

EDIT

Since someone downvoted the question without explanation, here is what I assume. In the particular example stated above, one could acutally draw 50 pictures. Visualizing only 16 filters (4x4) was choosen for convenience.

Answer 1

You're right, and there should be 50 images. You could easily verfiy this by:

[coef.shape for coef in mlp.coefs_[0]]

where mlp is the trained MLP classifer in the example.

So here are the two things caused confusion:

• Clearly, the author of the example did not mention anything about why only 16 images
• In Python when you zip two things in the for loop that don't have equal length (number of items) in this example zip(mlp.coefs_[0].T, axes.ravel()), Python will automatically ignore the extra items in the bigger lists (arrays, etc.). Here axes.ravel() has only 16 items, therefore, the loop iterates over first 16 vectors in mlp.coefs_[0]