I try to compare different CNN models. I use Keras and for training, I use a GPU, Google Colab with Tensorflow backend. Unfortunately I'm not able to create the same initial conditions for the CNNs (or in other words: I always get different results). Although by putting the following lines at the top of the code, I get always different results after every run.
from numpy.random import seed
seed(1)
from tensorflow import set_random_seed
set_random_seed(2)
Can it be that it is simply not possible to get reproducible results?
Would it be the best way to simply repeat the training several times and then either calculate a mean (if possible) or simply practice ensemble learning? All without using seeds or random_states or shuffle=False etc.
What would be the best way to compare these models?
tf.set_random_seed
(tf.random.set_seed
in TensorFlow 2) sets the seed for the default graph, but not for individual operations that might have their own random number generators $\endgroup$tf.nn.dropout
andtf.keras.layers.Dropout
accept aseed
parameter) as well as more subtle things like the random initialization of weights for dense or convolutional layers (e.g., you can pass aseed
parameter to the initializers in thetf.keras.initializers
module). $\endgroup$