3
$\begingroup$

I started coding backpropagation for a simple convnet and had some troubles understanding the algorithm. I do get the idea of weight update based on gradients, but because the filter kernel parameters are shared across the field, I am not sure hot to jointly process all gradients that should contribute to the update. How do I update the kernel values during backprop in this case?

$\endgroup$
2
$\begingroup$

Gradients flow back to where they came from. So any weight that contributed to an output gets updated with gradients from that output. In the case of a conv layer, a weight that contributed to multiple outputs get updated multiple times, once for each output. The updates are summed.

$\endgroup$
-1
$\begingroup$

Updating the weights (kernels) through the convolutional layer is done by convolving the (l+1) delta with the (l+1) weight. This gives the delta for updating the kernel. For a detailed explanation, check out deepgrid. Also, check out the github repo on CNN here, that implements backpropagation and updating the weights on a simple CNN.

$\endgroup$
  • $\begingroup$ Hi. We're looking for answers which can "stand alone" without requiring external links (links can die!). Could you please edit your answer to include the essential information in these links? Thanks. $\endgroup$ – Sycorax Jul 23 '18 at 23:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.