When training the Model on GPU, different results are retrieved for the same hyperparameters. This effect can be shut down by using CPU or Tensorflow 2.1. with deterministic settings.

The Post on ResearchGate "Whenever i run my neural network I get a different result" was answered with suggestions about random seeds and the usual. Yet one high rated comment by Sergey Porotsky stood out:

"It is good indicator, that control parameter values of your NN are not selected well. After good tuning, the difference (due to difference of the init random generator values) should be negligible. "

Can anyone confirm this statement? If so, which are the parameters that usually contribute to this kind of "not good tuned" fluctuation? These should not be included in the Hyperpatameter Search, should they?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.