6
$\begingroup$

Is there a theoretical basis against using both L2 and Dropout regularization simultaneously for training a deep neural network? They are both related but could they be complementary if used together?

$\endgroup$
9
$\begingroup$

The paper {1} that introduced dropout combined dropout with L2:

We found that dropout combined with max-norm regularization gives the lowest generalization error.

enter image description here


$\endgroup$
  • $\begingroup$ Thanks for the reference in the original paper! Sometimes it is best to go to the source. As a follow up, would there be an issue with using dropout + l2 + maxnorm simulteously? $\endgroup$ – Pylander Oct 19 '16 at 21:34
  • $\begingroup$ The link doesn't work anymore $\endgroup$ – Mayur Kulkarni Feb 16 '18 at 8:11
  • $\begingroup$ @MayurKulkarni Stack Exchange should take care of dead links, not individual users. $\endgroup$ – Franck Dernoncourt Feb 16 '18 at 8:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.