Is cross validation used for model selection only, or can I use cross validation to test the performance accuracy of a neural network?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Both! What changes is the way you apply it. This paper provides a good explanation of how to perform cross validation for estimating the classifier accuracy. First, notice that the term cross-validation refers to a set of procedures to estimate the performance of your model (LOOCV, holdout, k-Fold,...). See here for a shot. From now on, I assume you use k-Fold CV. It is found to perform better.
Beware that you need to make sure that you have enough data to apply it.
Now, if you additionally would like to perform model selection, then you need to perform a further split on the data. People usually refer to this as splitting the data in a training set, a validation set and a test set. The basic idea is that training an algorithm and evaluating its statistical performance on the same data yields an overoptimistic result.
If you have enough data, this suffices. Otherwise you need to resort to other techniques like stratified cross validation. This point is also referred in the previous paper by Kohavi.
