6
$\begingroup$

From what I can understand, it describes the phenomenon of when neurons detect the same features. Why does this happen?

$\endgroup$
  • 1
    $\begingroup$ From Hinton's paper: "complex co-adaptation is a phenomena where a feature detector is only helpful in the context of several other specific feature detectors." So it is not learning the same features. $\endgroup$ – yasin.yazici Mar 14 '16 at 7:49
2
$\begingroup$

Since the weights are not initialized properly and groups of neurons end up in the same local minima, according to their (similar) initialization.

To overcome this, you could use dropout / drop connect to break symmetry.

Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012). Improving neural networks by preventing co-adaptation of feature detectors.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ What does it mean for a neuron to be in a local minimum? $\endgroup$ – Matthew Drury Jun 2 '18 at 20:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.