4
$\begingroup$

Is it really necessary to use a validation set to avoide overfitting while we are using Dropout ?

$\endgroup$
4
  • $\begingroup$ Without comparison to a validation set, how would you know that you've successfully prevented overfitting? The probability of masking a unit can be anything between 0 and 1. Is masking masking probability $10^{-10}$ too small (model overfits)? Or too large (model underfits)? How can you know? $\endgroup$
    – Sycorax
    Commented Aug 11, 2020 at 15:23
  • $\begingroup$ I did not get the idea of masking units can u please expalin more $\endgroup$
    – Ka_
    Commented Aug 12, 2020 at 19:28
  • $\begingroup$ masking units at random is how dropout works. more information: stats.stackexchange.com/questions/241645/… $\endgroup$
    – Sycorax
    Commented Aug 12, 2020 at 19:29
  • $\begingroup$ So did u mean the validation set will help us to find the suitable dropout probability for our model $\endgroup$
    – Ka_
    Commented Aug 12, 2020 at 19:53

1 Answer 1

4
$\begingroup$

Like typical regularization, dropout is a mechanism to fight overfitting. It doesn't detect it. On the contrary, using a validation set helps you detect it. So, the two have different uses.

$\endgroup$
2
  • $\begingroup$ So if we are using Dropout we expect not having an over-fitted model.in this case do we really need to use a validation set ? $\endgroup$
    – Ka_
    Commented Aug 11, 2020 at 14:27
  • 2
    $\begingroup$ Dropout fights over-fitting, but it may not prevent it (it's not a silver bullet). Validation set is used for detecting/deciding the overfitting. So, you should use it nevertheless. $\endgroup$
    – gunes
    Commented Aug 11, 2020 at 14:40

Not the answer you're looking for? Browse other questions tagged or ask your own question.