Is the backpropagation (BP) algorithm the same for both fully-connected and locally-connected (or partially-connected) neural networks? I know how to use BP for a fully-connected network, but I don't know how to use BP for a locally-connected network. How would I calculate the derivative for those links which are not connected, is there any documentation for this?
$\begingroup$
$\endgroup$
asked Nov 17, 2014 at 8:39
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Yes, it's the same. A locally connected network can just be thought of as a fully connected with 0-valued weights for "distant" connections. In the backwards pass through these "non-connections", the gradient is multiplied by 0 and therefore ignored.
If your "local" connectivity is convolutional, then you pass back your gradient by reversing your kernel and doing another convolution. (This gives you the same result as if you considered it to have full connectivity with 0-weights for non-local connections).
