I have came across a work that did something in their cross validation protocol which I found strange: they run 3-fold cross-validation, but for each fold they used different hyperparameters during training. Specifically, the changed hyperparameters (to train their neural network) were:
- Number of training epochs.
- Use of early stopping - for one of the folds they pick the epoch with best validation performance among 300 epochs, while the other two folds run for 150 and 500 epochs respectively, and they just pick the model of the last epoch.
I don't recall ever seeing this sort of thing for cross-validation protocols, so I am wondering if this kind of protocol is outright wrong or just weird.