How does local connection implied in the CNN algorithm

Question

I am trying to understand the process of Convolutional Neural Networks. Basically, I am trying to understand how does the local connection works. The first step of CNN is a convolution layer where every image is convolved with filters. If for example I have 100 filters, how does the local connections working? If I am understanding well, I have to convolve the input image with all 100 filters in order to produce 100 feature map in the convolutional layer. How does local connection implied in the CNN process? Where is the idea of local receptive fields implied?

EDiT: After the design of the architecture. The weights of all layers, convolutional pooling and the fully connected layer are trained by using back propagation?

Answer 1

To understand the local connectivity, first think about giving an image as input into just a regular fully connected neural network. Each input (pixel value) is connected to every neuron in the first layer. So each neuron in the first layer is getting input from EVERY part of the image.

With a convolutional network, each neuron only receives input from a small local group of the pixels in the input image. This is what is meant by "local connectivity", all of the inputs that go into a given neuron are actually close to each other.

For your second question, yes, both the fully connected layers and the convolutional layers can be trained using back propagation. You take the errors after propagating back to the first fully connected layer and start your convolutional layer propagation using those.

Answer 2

Imagine it's a digit 7 in the image, which is 4*4 image.

We need to classify the digit in the image from digits 0-9.

Consider breaking the image into 4 regions. Here, color coded as red,green, yellow and blue. Then, each hidden node could be connected to only the pixels in one of these 4 regions, each hidden node sees only a quater of the original image.

With this new regional breakdown and the assignment of small local groups of pixels to different hidden nodes, every hidden node finds patterns in only one of the four regions in the image.

Then, each hidden node still reports to the output layer where the output layer combines the findings for discovered patterns learned separately in each region.

This is called local connected layers.

Answer 3

I hope this image will help you to understand the idea of local connectivity in CNN:

.

Neurons of the same color use the same kernel filter and share the same weights. And, neurons of different color corresponds to different feature maps. So, here we have local connectivity since one blue neuron is connected to only a small region of the image. (P.S. to produce a feature map, you need more than one blue neuron).

Answer 4

It's often poorly explained. First of all, it's presented in terms of a 4D tensor, but one dimension is just the batch dimension for processing multiple images at a time, you can ignore it for the purpose of understanding the convolution.

So images in a traditional CNN are three dimensional, channels x height x width The filters are four dimensional and have the structure: input_channels x height x width x output_channels. You can think of them as several (#output_channels of them) 3D linear filters applied to the image.

In this image, the green box is a filter, it convolves by "gliding" in the x and y axis, but its height (#input_channels) is the same as the number of channels in the image, and so it does not move in that direction.

There are #output_channels such boxes which end up producing the channels of the next layer.