I have a recurrent LSTM network, which classifies text and I want to find the best hyperparameters. I use GridSearchCV from scikit-learn and a classifier, which uses generators to transform my sparse vectors to dense vectors.
While I can execute the grid search, i have to define the number of epochs first. Furthermore the default implementation uses 3-fold cross validation. As scikit-learn does manage the split of my dataset I can only get the validation score after all epochs (intead of after each epoch).
My problem is that my network does converge fast to 90% accuracy in the most cases but varies from epoch to epoch in the end about 5%. So my grid search results are not really reliable (uses what is the score after x epochs).
Before I started with grid search I just used a train test split and compared parameters by using the maximum achieved accurancy on the validation data, which was scored after each epoch. In this picture you can see the loss over 20 epochs.
My question is: Does it make sense to use grid search with cross validation here, or would you use the train/test split and determine the best hyperparameters, by looking at the best accuracy achieved after each epoch.