I understand how k-fold cross validation works. For each iteration, the training data is split into $k$ portions, and using $k-1$ portion of the data for training and the $k$-th portion of the data for testing.
Can we relate k-fold cross validation to the idea of empirical-risk minimisation ? where suppose we are able to define a probability distribution over the $(x,y)$ training examples. At each cross validation iteration, we sample $(k-1)$ portions of the $(x,y)$ data from $p(x,y)$.
Intuition tells me that since we are sampling many times from $p(x,y)$ for each iteration and computing validation error, it would result in a lower variance for the validation error.
Is that the main benefit of k-fold cross validation ?