I am studying convolutional neural networks (CNNs) due to their applications in computer vision. I am already familiar with standard feed-foward neural networks, so I'm hoping that some people here can help me take the extra step in understanding CNNs. Here's what I think about CNNs:
- In traditional feed-foward NNs, we have training data where each element consists of a feature vector that we input to the NN in the "input layer," so with image recognition, we could just have each pixel be one input. Those are our feature vectors. Alternatively, we could manually create other -- likely smaller -- feature vectors.
- The advantage of the CNN Is that it can generate stronger feature vectors that are more invariant to image distortion and position. As the following image shows (from this tutorial), CNNs generate feature maps that are then fed to a standard neural network (so really it's a giant pre-processing step).
- The way we get those "better" features is by alternating convolution and sub-sampling. I understand how sub-sampling works. For each feature map, just take a subset of the pixels, or we can average out values of pixels.
But what I'm mainly confused on is how the convolution step works. I am familiar with convolutions from probability theory (density for the sum of two random variables), but how do they work in CNNs, and why are they effective?
My question is similar to this one but in particular, I am not sure why the first convolution step works.