1
$\begingroup$

When writing a deep learning paper, I need to train several CNN models and compare their performances. They are from different architectures so different designs.

I'm wondering should I use the same learning rate for all models when training (I've made sure they all have the same batch size, same loss function, etc), or should I create a naive tuner and find the best learning rate for each of the model?

Thank you in advance!!

$\endgroup$

1 Answer 1

1
$\begingroup$

You cannot rule out that different architectures will benefit from different learning rates, so if you have the resources, you should give it a try.

Just make sure that you have a decent optimizer and that you have followed all the standard rules for learning DNNs.

$\endgroup$
2
  • $\begingroup$ Thank you! So if I have to use different learning rate for each one, do I need to indicate what I used for each one in the paper, or do you think I don't include such detailed information like the learning rate in my paper? Thank you again! $\endgroup$
    – Dwa
    Aug 25, 2022 at 15:08
  • 1
    $\begingroup$ It doesn't hurt to add it. Maybe you provide an appendix for all the details. $\endgroup$
    – frank
    Aug 25, 2022 at 15:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.