I am trying to learn convolutional neural networks from scratch.
I try to find a simple example that I can calculate by hand just to get the ideas.
There are many things that I do not understand so let's start simple.
I found some CNN examples that detect shapes like X, O, /, \ or faces like :) and :(, so just stuff that you can draw in a frame with 8x8 boxes.
In many examples filters are already given, but as I know filters are "trained" in the hidden layer via backpropagation.
So a CNN starts with filters with random values but I do not understand how the filters become what they are, I mean how a filter A becomes a detector for straight lines, or how a Filter B becomes a detector for curves. I just know that with some "magic" you end up with different filters and each filter is unique and detects a special shape/feature.
The other thing is who is telling me how many filter the CNN should have? Is it just intuition or just testing?
Whats about the filter size, in many example I saw that the dimensions of the filters 3x3, but why not 5x5 or other dimensions?
What would really help me is a small and simple step by step example.
Thanks for respond Firebug!
There are many things that I still do not get and that I have to study. For example I did not know that deep learning is largely based on heuristics/try and error.
I created a little example that I found and as you can see its very simplified
In this examples the filters are given and the weights are manually changed so that we get an average of 0.8 for all votes. As far as I know I can decide what accuracy I accept, in this case we just decide that 80% is enough.
What really would help me is to see a step by step backpropagation by hand with all kernels and weights set to 1 so I can see how the kernel and the weights get the values that they get. I would also like to see the cost-function that is optimized.
It is very abstract for me at the moment but I hope you can help me out, then I can move on to the other details.