Having studied ordinary fully connected ANNs, I am starting to study convnets. I am struggling to understand how hidden layers connect. I do understand how the input matrix forward feeds a smaller field of values to the feature maps in the first hidden layer, by moving the local receptive field along one each time and forward feeding through the same/shared weights (for each feature map), so there are only one group of weights per feature map that are of the same structure as the local receptive field. Please correct me if I am wrong. Then, the feature maps use pooling to simplify the maps. The next part is when I get confused, here is a link to a 3d CNN visualisation to help explain my confusion

http://scs.ryerson.ca/~aharley/vis/conv/

Draw a number into the top left pad and you'll see how it works. Its really cool. So, on the layer after the first pooling layer (the 4th row up containing 16 filters) if you hover your mouse over the filters you can see how the weights connect to the previous pooling layer. Try different filters on this row and what I do not understand is the rule that connects the second convolution layer to the previous pool layer. E.g on the filters to the very left, they are fully connected to the pooling layer. But on the ones nearer to the right, they only connect to about 3 of the previous pooled layers. Looks random.

I hope my explanation makes sense, if not please do ask further. I am essentially confused about what the pattern is that connects hidden pooled layers to the following hidden convolution layer. Even if my example is a bit odd, I would still appreciate some sort of explanation or link to a good explanation.

Thanks a lot.

  • Love the visualization in the link you send. If i would have to venture a guess as to why some of the nodes are not fully connected, it might be that pruning is applied at some point in the training algorithm. I.e. weights that are near zero are removed to improve generalization. – GR4 Apr 1 '17 at 11:05
  • That makes sense. Otherwise, would it be fully connected? – harry lakins Apr 1 '17 at 11:06
  • To my knowledge yes, but am honestly not sure. (Also, I think we probably shouldn't be referring to this as fully connected as it might confuse with a fully connected layer. ) – GR4 Apr 1 '17 at 11:52
  • Well in that case surely it would be fully connected because every weight does communicate with every neutron even if they are shared weights – harry lakins Apr 1 '17 at 11:55

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.