I understand how k-fold cross validation works. For each iteration, the training data is split into $k$ portions, and using $k-1$ portion of the data for training and the $k$-th portion of the data for testing.

Can we relate k-fold cross validation to the idea of empirical-risk minimisation ? where suppose we are able to define a probability distribution over the $(x,y)$ training examples. At each cross validation iteration, we sample $(k-1)$ portions of the $(x,y)$ data from $p(x,y)$.

Intuition tells me that since we are sampling many times from $p(x,y)$ for each iteration and computing validation error, it would result in a lower variance for the validation error.

Is that the main benefit of k-fold cross validation ?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.