3
$\begingroup$

Is the backpropagation (BP) algorithm the same for both fully-connected and locally-connected (or partially-connected) neural networks? I know how to use BP for a fully-connected network, but I don't know how to use BP for a locally-connected network. How would I calculate the derivative for those links which are not connected, is there any documentation for this?

$\endgroup$

1 Answer 1

3
$\begingroup$

Yes, it's the same. A locally connected network can just be thought of as a fully connected with 0-valued weights for "distant" connections. In the backwards pass through these "non-connections", the gradient is multiplied by 0 and therefore ignored.

If your "local" connectivity is convolutional, then you pass back your gradient by reversing your kernel and doing another convolution. (This gives you the same result as if you considered it to have full connectivity with 0-weights for non-local connections).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.