Does training time of CNN includes the validation time performed after every epoch?

Question

I want to record train time of a CNN architecture in keras. My question is that while we are training a CNN, we also validate the model after every epoch to monitor that how well it generalizes which actually gives us a pretty good idea about our model performance. Now, if I have to report training time of this model, will I be considering only the training time or I have to include the validation time too. Secondly, the training time reported in published literature of CNN, does it include the validation time after every epoch or just the train time without any validation on a separate dataset (other than train)?

Answer 1

Now, if I have to report training time of this model, will I be considering only the training time or I have to include the validation time too.

If you are using validation just to see the performance on unseen data you can remove that step from your training loop and achieve identical results. So to that extent the time spent on validation should not be included.

However if you are performing early stopping it might be argued that the validation serves an integral purpose during training to determine which epoch generalises best to unseen data. On the other hand, early stopping can also be seen as a form of hyperparameter tuning (i.e. given constant seeds what is the ideal number of epochs), and I have never seen the time taken for hyperparameter tuning included in literature.

So if you aren't using early stopping don't include the validation time. If you are using early stopping I'd say include the validation time but also report the time excluding validation. There is no disadvantage to reporting multiple times calculated in different ways and explaining what each time involves.

I cannot comment on what the reported times in literature involve, so hopefully someone else with have more insight on that part of the question.