# Cross validation Deep learning

$$f_{i}=A(D/ D_{i})$$ $$e_{j}=L(f_{i}, z^{(j)})$$

It seems to me, that above definition of k-folded cross validation algorithm (from Deep Learning book by Ian Goodfellow and Yoshua Bengio and Aaron Courville, 2016) is inconsistent with the common definition of cross - validation. In above algorithm $e$ vector is the vector of loss function calculated for every particular example in the $D$ dataset, and then mean of vector $e$ is the estimation of generalization error. Whereas in standard definition of cross - validation, we calculate test error for each fold and then calculate average of them.

• What is your question? Feb 10 '17 at 19:37
• What is the difference between those two definitions of cross validation Feb 10 '17 at 19:39

The algorithm returns the vector of errors $\epsilon$ for each example in $\mathbb{D}$, whose mean is the estimated generalization error.
To square this with the definition you give above, note that the mean of all errors is equal to the weighted (by $\frac{|\mathbb{D_i}|}{N}$) average of means over each of k folds. (K-fold cross validation error is a weighted average of errors of particular folds.)
• Sorry, I goofed that formula twice—should now follow from expressing each group mean as $\Sigma_{\mathbb{D}_i}\epsilon_j$. Feb 10 '17 at 20:44