0
$\begingroup$

I would like to compare the classification performance (like accuracy, precision, recall etc.) of different CNN architectures. I'm using Google Colab (GPU support), Tensorflow and Keras. Since it is unfortunately not possible to make the trained models reproducible, I would like to train each CNN architecture several times and then calculate the mean and standard deviation of the results like accuracy, precision, recall etc.

It is advisable to set different seeds for the random-generators in each run like follows?

import os
os.environ['PYTHONHASHSEED']='0'
import numpy as np
import tensorflow as tf
import random as rn
np.random.seed(1)
rn.seed(1)
from keras import backend as K
if 'tensorflow' == K.backend():
   import tensorflow as tf
   from keras.backend.tensorflow_backend import set_session
   config = tf.ConfigProto()
   config.gpu_options.allow_growth = True
   config.gpu_options.visible_device_list = "0"
   set_session(tf.Session(config=config))
tf.set_random_seed(1) 

or is it better to use no seeds at all?

What would you recommend?

$\endgroup$
  • 1
    $\begingroup$ Mean and standard deviation of what? And why? What do you want to do with them? What problem are you trying to solve? It's impossible to determine what is advisable or better or recommended without knowing what your goals and requirements are and what problem you are trying to solve. If you are taking the mean and standard deviation of the weights of the CNN, I don't expect any reason for that to be meaningful or useful. $\endgroup$ – D.W. Nov 3 '19 at 18:48
  • 2
    $\begingroup$ Please don't leave clarifications in the comments. Instead, edit the question so it contains all relevant information and reads well for someone who encounters it for the first time, then flag comments as 'no longer needed' once you have addressed all the feedback in them. People shouldn't have to read the comments to understand your question. Thank you! $\endgroup$ – D.W. Nov 3 '19 at 19:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.