Suppose a DNN, a parametric model with weights trained via gradient descent.
- Shuffle dataset
- Split into 10 equal sets
- Train on sets 1-9, validate on 10
- Reinitialize model
- Train on sets 2-10, validate on 1
- Repeat until all 10 combinations were trained/validated on, then average validation results
B. "A" except skip step 4, and we change splits every epoch rather than fitting a split until completion (e.g. early stopping). So train on 1-9, validate on 10, then continue with same model and its weights to train on 2-10, etc.
"A" results in 10 different weights but same architecture and hyperparameters, "B" results in only one model and one weights. "A" validates on unseen data, "B" validates on fitted data.
Which one is K-Fold Cross Validation?