I trained CNN to over-fit the data (close to 0 train error), it took 6 days to train. Now I want to regularise the model using L2 or Dropout, with weights initialised from over-fitted model, will it work? or should start training again from scratch?

  • $\begingroup$ “Will it work”—will it run? Yes. Will it perform better than it does already? Not necessarily. Are you asking which initialization will perform better with the same training regimen? $\endgroup$ Jun 3, 2021 at 3:05
  • $\begingroup$ yes, which will give better test performance? $\endgroup$
    – spb
    Jun 3, 2021 at 3:18
  • $\begingroup$ There's no hard-and-fast rule here. $\endgroup$ Jun 3, 2021 at 3:27


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.