Evaluation network performance using cross-validation

Question

Suppose I have a data set on which I'm training a neural network. I'm using four-fold validation, meaning that I train four models, one for each fold. Two of the folds are used for training, one for testing and one for validation so that no model is trained on its testing data.

The result of this process is four trained models (obviously). My question is what is the correct way of reporting the performance for my network? Should I report the arithmetic mean of these four models on their respective folds or something else?

Answer 1

Reporting the arithmetic mean and the standard deviation of the four models on their respective test sets is the usual way of performance reporting, so you can do that.

Just keep in mind if you tune hyperparameters (e.g. you are comparing several different network architectures, use different learning rates, etc.), you should not use these cross validation results to pick the final model - you would be overfitting the hyperparameters to your dataset. For this purpose you will need to keep aside an additional test set.

See also this related answer.