2
$\begingroup$

When using leave one out cross validation in neural network, do I have to fix the epoch number for each training model?

The test results of these models are averaged to show performance. So can I choose best result for each model (the epoch number will be different) or do I have to fixed epoch number for each model (the epoch numbers will be same but the results are not all best)?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
4
$\begingroup$

I've never seen any constraint on the number of epochs, but I think it doesn't make much sense... You can try setting a desired training error e and a maximum number of epochs n. If n is reached before e is achieved, stop the training and start testing, and the lowest error will be reported as the training error of the model. You train with 9 folders, and test with the other one.

I think you might be confused about training and test error. Training error is achieved iteratively, the parameters are changing; testing error is achieved with fixed parameters, there is no epoch number for testing error.

Nevertheless, always make it clear how you're doing the training/testing!

Improve this answer
$\endgroup$
1
$\begingroup$

From your question you have a neural network training method that comes with a criterion for determining if the learning has converged enough. If you fix the epoch number you won't be validating this convergence criterion. Hence, it seems to me, you do not want to fix the epoch number.

Improve this answer
$\endgroup$
3
  • $\begingroup$ hi, thanks for your reply. I actually want to compare with previous result from a paper. The paper said they use leave one out cross validation to measure the performance. There are 10 folds. Each time train 9 sets and test 1 set. After training and testing 10 times, the results are averaged to be as the performance. So do you mean that the training epoch number can be different for each model so that their test results can be best? $\endgroup$
    – kelly1221
    Commented Jul 25, 2015 at 8:56
  • $\begingroup$ You are not doing leave one out cross validation if you use 10 folds (unless you also only have 10 points in your training data). I do indeed mean that the epoch number can be different. $\endgroup$
    – kasterma
    Commented Jul 25, 2015 at 9:53
  • $\begingroup$ I mean I have 10 sets of data, and each time train 9 sets and validate 1 sets. there are no other test data. I will obtain 10 models (each is trained by different 9-training sets and tested different 1-set). And the performance will be averaged by these 10 test results. So there is no need to make constraints that there 10 models should have the same training epoch number? $\endgroup$
    – kelly1221
    Commented Jul 26, 2015 at 2:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.