I am planning to use a $10$-fold strategy in order to validate the results of a logistic regression analysis.
I am a bit confused about the procedure I must follow.
More precisely: Let us suppose that my cross-validation returns acceptable values (for the average AUC, average misclassification rates, etc.) and reveals that everything is OK. Which is the predictive model or formula I should return in such a case?
First of all, I have already read (here, for instance: With k-fold cross-validation, do you average all $k$ models to build the final model?) and understood that a $k$-fold cross-validation procedure aims at assessing the performance of a predictive analysis on a given data set (in terms of its ability to accurately predict on new data). I mean, it is not a technique for building predictive models, but a way to measure whether a given fitted model can be overfitting or not (roughly speaking).
OK, so, as far as I understand, I must do this:
Randomly divide the sample into $k$ equally sized subsamples (using stratification if necessary).
For $i=1$ to $k$:
- Perform a logistic regression analysis using all the cases not in subsample $i$ as the training set.
- Use subsample $i$ as the validation set. Calculate performance parameters.
- Calculate average performance parameters.
Let us imagine that step 3 allows us to conclude that logistic regression performs well on our data.
Now, I have obtained $k$ predictive models. Is this right?
So, which one of them is the final model I should use? Or, should I build (and return to the final user) a definitive logistic regression predictive model using all cases in the sample?
Sorry if this is too simple or too wrong. I would really appreciate if you can answer my questions.
Actually, my question is about cross-validation, but I wanted to specify that I am dealing with logistic regression, just in case it helps.