9
$\begingroup$

Based from what I have learned, we use multiple filters in a Conv Layer of a CNN to learn different feature detectors. But since these filters are applied similarly (i.e. slided and multiplied to regions of the input), wouldn't they just learn the same parameters during training? Hence the use of multiple filters would be redundant?

$\endgroup$
6
$\begingroup$

I had the same confusion in understanding this fact. The confusion arises to the beginners because the book explicity doesn't mention that filters are different.

since these filters are applied similarly

Filters are applied similarly but the value of cell in matrix is different from each other filters. So they extract different features from the image.

wouldn't they just learn the same parameters during training

No, they don't learn the same parameter since the filters are different now. So the use of multiple filter is not redundant.

$\endgroup$
  • $\begingroup$ Thank you for answering. What exactly makes them different? How do we ensure that they learn different parameters during training? Is it their initial values? $\endgroup$ – cjbayron Jun 16 '18 at 9:24
  • 1
    $\begingroup$ different Values of each cell make them different. Like some will detect slant line, some will detect 45 degree curve line etc. So they all are different. $\endgroup$ – ironman Jun 16 '18 at 9:27
  • $\begingroup$ Yes i understand that different values make the filters detect different features. But how do these filters learn differently during training? $\endgroup$ – cjbayron Jun 16 '18 at 9:31
  • 1
    $\begingroup$ If the filters all start off the same then they will stay that way. The random initialization means they start off different and from there they learn different things. Look up symmetry breaking in neural networks for more information. $\endgroup$ – Aaron Jun 19 '18 at 16:12
2
$\begingroup$

I have found the answer to this question: https://www.quora.com/Why-does-each-filter-learn-different-features-in-a-convolutional-neural-network

It says here: "... (optimization) algorithm finds that loss does not decrease if two filters have similar weights and biases, so it’ll eventually change one of the filter(‘s weights and biases) in order to reduce loss thereby learning a new feature."

Thank you for the answers. Appreciate it :)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.