I have got a good grasp of backpropagation algorithm for fully connected neural networks and I can derive the vectorized implementation for it but after getting into convulutional networks, I think due to my not-so-strong linear algebra background, I have a lot of trouble trying to come up with a fully vectorized implementation for backpropagation.

The network architecture is as follows :-


I have used only one convolutional Layer for simplifaication

How can the vectorized implementation of this network be derived ?

  • $\begingroup$ Where are you stuck? $\endgroup$
    – Aaron
    Aug 15, 2017 at 16:37
  • $\begingroup$ @Aaron Well I know the formula for convolution so I can calculate element-wise derivative but I want to vectorize it. $\endgroup$ Aug 16, 2017 at 1:09
  • $\begingroup$ Start with one-dimensional case with only 1 variable. Derive derivatives. Extend to slightly harder case, again, derive derivatives, combine them into vector form (should be easy once correct derivatives are computed); alternatively you can lookup a the answer in the book. Then move to the most general case which is for images. $\endgroup$
    – Gnattuha
    Aug 20, 2017 at 19:14
  • $\begingroup$ @Gnattuha I have read an article and understood most of the derivation but I'm stuck . here are more details about my problem. Please help. $\endgroup$ Sep 2, 2017 at 8:36

1 Answer 1

  1. Convert all kernels to columns and get a kernels matrix
  2. Split your input (image) on slices for convolution then convert to columns and get an input matrix. You can append other inputs (images) to form a mini-batch
  3. multiply transposed input matrix on kernels matrix. In the result matrix each column is one feature map.

Look here for more details


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.