I have a deep neural network model and I need to train it on my dataset which consists of about 100,000 examples, my validation data contains about 1000 examples. Because it takes time to train each example (around 0.5s for each example) and in order to avoid overfitting, I would like to apply early stopping to prevent unnecessary computation. But I am not sure how to properly train my neural network with early stopping, several things I do not quite understand now:
What would be a good validation frequency? Should I check my model on the validation data at the end of each epoch? (My batch size is 1)
Is it the case that the first few epochs might yield worse result before it starts converging to better value? In that case, should we train our network for several epochs before checking for early stopping?
How to handle the case when the validation loss might go up and down? In that case, early stopping might prevent my model from learning further, right?
Thank you in advance.