7
$\begingroup$

For building deep neural networks, there are a lot of random components in each training. On one hand, I feel it is uncanny to "tune" random seed. But in my experience, some random seed just works better than others ...

So, Is random seed a hyper-parameter to tune in training deep neural network?

Improve this question
$\endgroup$
3
  • 5
    $\begingroup$ Whenever I play blackjack, I like to "tune" the random seed of the dealer so that I get to 21 in the first few cards. $\endgroup$
    – Sycorax ♦
    Commented Jul 21, 2020 at 12:40
  • $\begingroup$ @Sycorax thanks for your comments but I am not sure I fully understand your analogy or joke... $\endgroup$
    – Haitao Du
    Commented Jul 21, 2020 at 12:52
  • $\begingroup$ stats.stackexchange.com/questions/341619/… $\endgroup$
    – Tim
    Commented Jul 21, 2020 at 13:31

1 Answer 1

Reset to default
9
$\begingroup$

If your training has large changes in performance due to the random seed, then it is unstable. This is an undesirable trait. Testing different random seeds can in this sense be useful to check stability. But picking a model from a given random seed which happens to do better on the validation set does not guarantee better performance on unseen data.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ We can tune seed on validation and report the result on the test set. Similart to other hyperparameters that are optimised on the validation set. $\endgroup$
    – keramat
    Commented Oct 25, 2021 at 15:16
  • $\begingroup$ Why do you think that would give better performance on the test set? IE, why would it not be just overfitting to the validation set? $\endgroup$
    – Jon Nordby
    Commented Nov 21, 2022 at 19:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.