I would like to compare the classification performance (like accuracy, precision, recall etc.) of different CNN architectures. I'm using Google Colab (GPU support), Tensorflow and Keras. Since it is unfortunately not possible to make the trained models reproducible, I would like to train each CNN architecture several times and then calculate the mean and standard deviation of the results like accuracy, precision, recall etc.
It is advisable to set different seeds for the random-generators in each run like follows?
import os os.environ['PYTHONHASHSEED']='0' import numpy as np import tensorflow as tf import random as rn np.random.seed(1) rn.seed(1) from keras import backend as K if 'tensorflow' == K.backend(): import tensorflow as tf from keras.backend.tensorflow_backend import set_session config = tf.ConfigProto() config.gpu_options.allow_growth = True config.gpu_options.visible_device_list = "0" set_session(tf.Session(config=config)) tf.set_random_seed(1)
or is it better to use no seeds at all?
What would you recommend?