0
$\begingroup$

I am trying to understand the logic behind convolutional neural networks. To my understanding, the weights used are nothing more than an $w \times h$ matrix (a filter) and as with the normal neural networks, those filters are randomnly initialized.

My question is how are they updated? Is an optimization algorithm used to do so (e.g. SGD)? If yes, at which step of training is this happening (is there a backoprop step as in common neural networks)?

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ They are updated exactly the same as any other parameters. Could you tell us what is your understanding and where are you lost? $\endgroup$
    – Tim ♦
    Dec 11, 2018 at 15:16
  • $\begingroup$ I'm lost because in most of the sources I've seen, they only discuss about initializing the filters and they seem to not care about their values during the backprop step. $\endgroup$
    – kingJulian
    Dec 11, 2018 at 15:24

1 Answer 1

Reset to default
1
$\begingroup$

Yes, they are updated using SGD (or SGD variants: RMSprop, Adam, etc) through backpropagation. To see this working with great animations and numerical examples (with code in python/numpy) I suggest you to see this amazing post by Andrej Karpathy: http://cs231n.github.io/convolutional-networks/

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.