I'm a bit confused about the visualization of the weights of a feed forward neural network as provided in this example from scikit-learn. The network has an architecture of [784 x 50 x 10] (MNIST dataset), so n_hidden = 50 (there are 50 hidden units). Each of the hidden units is a column of the weight matrix W_1 (with dimension 784x50), right? From this, I thought, we could create 50 images, each of size 28x28 (=784). However, there are only 16 images shown? What is my mistake here?

enter image description here


Since someone downvoted the question without explanation, here is what I assume. In the particular example stated above, one could acutally draw 50 pictures. Visualizing only 16 filters (4x4) was choosen for convenience.

  • $\begingroup$ Interesting question. But I have a question for you too: have you run the code yourself too? The author of the article might have chosen to show only 16/50 neurons. So if you didn't run the script yourself: do so! $\endgroup$ – Thomas W Apr 25 '17 at 15:26
  • $\begingroup$ @Thomas W, I did and came to the same conclusion as you. Just really thought I misunderstood something and went over and over again. There is a similar example on the theano tutorial side where they just visualized 100/500 neurons without further explanation. $\endgroup$ – Tim Apr 26 '17 at 20:35

You're right, and there should be 50 images. You could easily verfiy this by:

[coef.shape for coef in mlp.coefs_[0]]

where mlp is the trained MLP classifer in the example.

So here are the two things caused confusion:

  • Clearly, the author of the example did not mention anything about why only 16 images
  • In Python when you zip two things in the for loop that don't have equal length (number of items) in this example zip(mlp.coefs_[0].T, axes.ravel()), Python will automatically ignore the extra items in the bigger lists (arrays, etc.). Here axes.ravel() has only 16 items, therefore, the loop iterates over first 16 vectors in mlp.coefs_[0]

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.