It seems to a relative novice like myself that effective use of the more thorough hyperparameter tuning functions (GridSearchCV(), RandomizedSearchCV(), etc.) is stymied by the stochastic nature of deep learning, where so much depends upon randomly initialized weights. In other words, these tuning functions will select entirely different sets of hyperparameters as optimal depending on randomly selected weights at initialization.
How does one most effectively confront this issue of randomness to ensure that when one DOES tune hyperparameters with one of the above functions, it isn't a complete waste of time?
I had thought about setting a random seed until I obtained decent results on a test set (I would be using a time series split), and only then using RandomizedSearchCV(), as at least I would know that at least one of the parameter configurations to be tested by RandomizedSearch was capable of producing decent, cross-validated results, but I don't know if this is an effective way, or if a better way exists.