My past assumption of cross-validation (in particular k-fold CV) was that in order to given same chance to each sample in our dataset to appear in training , we use k-fold CV. Under my assumption we define our classifier/model (select the type of classifier/model and set the classifier/model parameter) and test the performance of it on different test dataset by using k-fold, but after reading some posts, I found that we use k-fold CV in order to choose which one of our trained classifier/model is best and so we would choose it as our final classifier/model (which set of model parameter lead to better performance).
So now I have a these two questions:
First, which one of this is correct?
in k-fold CV, our classifier/model is set, and we only use k repetition to obtain better (more accurate) evaluation of our classifier/model performance (for example accuracy).
in k-fold CV, our classifier/model is not set (the type of classifier/model is selected (for example LDA) but the selected classifier/model's parameter are not set), and we only use k repetition to identify which classifier/model lead to better performance so we select that one as our final classifier/model. The performance measurement is not the main focus here.
Second, if the second definition is correct one, how can we explain the final averaging in k-fold cross-validation: how can a person average different classifier/model (a same classifier/model with different parameter value)?