I have this code, with the aim being to develop a neural network with cross validation and hyperparameter optimisation for a regression problem (continuous features, continuous label).

from keras.layers import Dense, Activation
from keras.models import Sequential
from sklearn.model_selection import StratifiedKFold

def create_model(activation='relu',learning_rate = 0.01):
  model = Sequential()
  model.add(Dense(32, activation = activation, input_dim = 101))
  model.add(Dense(units = 64, activation = activation))
  model.add(Dense(units = 64, activation = activation))
  model.add(Dense(units = 1))
  model.compile(optimizer = 'adam', loss = 'mean_squared_error')
  return model

model = KerasRegressor(build_fn = create_model,
                       verbose = 1)

params = {'activation': ["relu", "tanh"],
          'batch_size': [16, 32, 64], 
          'epochs': [50, 100],
          'learning_rate': [0.01, 0.001, 0.0001]}

random_search = GridSearchCV(model,param_grid = params, cv = KFold(5))
random_search_results = random_search.fit(X_train, y_train)
print("Best Score: ",random_search_results.best_score_, "and Best Params: ", random_search_results.best_params_)

Can someone confirm that if my next line is:

y_pred = random_search_results.predict(X_test)

That that is fitting the most optimal model to X_test? I thought it was, but then I saw this post, where they say 'The next task is to refit the model with the best parameters', after the code above is run.

Did I not already fit the best param model to the training data using this method? Can someone explain to me what extra code is needed to add the optimal model according to GridSearchCV to the training data?


Hello and welcome to the community :-)

GridSearch searches the best estimator. Period. Thats the fundamental difference between RandomizedSearchCV and GridSearchCV ... and why GridSearch takes so awkwardly long.

It may be that you will get slightly different params when using different random states, but all in all a pipeline and the hyperparameter tuning is just for finding your optimal combination of parameters.

After that you tak these combinations AND FIT

1.) on the whole data for deployment

2.) Train_data for deployment

The latter makes sense, if data is massive and neural network is so complex that training takes a considerable amount of time (e.g. imagine you get new data for a complex NN and get new data points e.g. 1.000.000 you won't fit your model every week, that too exhaustive.) and you dont want to get the well other 10-20% also into the set, that is if you have e.g. 1.000.000 data points, the other few percent wont budge the model at the end so hard, that train data may be enough, but it depends on the case/model.

So all the guy does is using the optimal combination and fitting on the whole data, which is some sort of get my final results for deployment.

If you predict with this line, you will get the best predict from gridsearch +/- a slightly deviation depending on random_state in the previous pipeline


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.