3
$\begingroup$

enter image description here

$$f_{i}=A(D/ D_{i})$$ $$e_{j}=L(f_{i}, z^{(j)})$$

It seems to me, that above definition of k-folded cross validation algorithm (from Deep Learning book by Ian Goodfellow and Yoshua Bengio and Aaron Courville, 2016) is inconsistent with the common definition of cross - validation. In above algorithm $e$ vector is the vector of loss function calculated for every particular example in the $D$ dataset, and then mean of vector $e$ is the estimation of generalization error. Whereas in standard definition of cross - validation, we calculate test error for each fold and then calculate average of them.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ What is your question? $\endgroup$
    – Sean Easter
    Feb 10, 2017 at 19:37
  • $\begingroup$ What is the difference between those two definitions of cross validation $\endgroup$
    – mokebe
    Feb 10, 2017 at 19:39

1 Answer 1

Reset to default
7
$\begingroup$

There's really no distinction between those definitions. (Perspective, perhaps?) As the pictured text reads, emphasis mine:

The algorithm returns the vector of errors $\epsilon$ for each example in $\mathbb{D}$, whose mean is the estimated generalization error.

To square this with the definition you give above, note that the mean of all errors is equal to the weighted (by $\frac{|\mathbb{D_i}|}{N}$) average of means over each of k folds. (K-fold cross validation error is a weighted average of errors of particular folds.)

Improve this answer
$\endgroup$
4
  • $\begingroup$ "note that the mean of all errors is equal to the weighted (by |Di||Di|) average of means over each of k folds.". Could you provide derivation? $\endgroup$
    – mokebe
    Feb 10, 2017 at 20:11
  • $\begingroup$ Sorry, I goofed that formula twice—should now follow from expressing each group mean as $\Sigma_{\mathbb{D}_i}\epsilon_j$. $\endgroup$
    – Sean Easter
    Feb 10, 2017 at 20:44
  • 1
    $\begingroup$ My misconception was a result of assumption that cross validation error is a non - weighted average of errors of particular folds, wheareas cross validation is a weighted average, which is often not highlighted in textbooks. $\endgroup$
    – mokebe
    Feb 10, 2017 at 21:38
  • $\begingroup$ Often times texts will sort of hand-wave over the assumption that each of the k folds are of equal size (making the two equivalent). Will jot up an edit shortly. $\endgroup$
    – Sean Easter
    Feb 10, 2017 at 21:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.