# Leave one out cross validation and set seed in R

I am having trouble to understand where and how many times to use set.seed().

I am performing a leave-one out crossvalidation on a set with 120 classified variables to predict another 300 of unknown class. Should I set the seed at the beginning of the code or at the beginning of each iteration of the leave-one-out cross validation in order to guarantee a run that can be repeated?

Thanks for any help and explanations.

## 2 Answers

If you set it once, the run will be replicable. Setting it multiple times will, at best, just slow things down. At worst, it would no longer be random. (It's hard to tell without your actual code).

Are you really doing leave-one-out cross validation?

Leave-one-out cross validation is usually complete in the sense that one iteration (consisting of $n$ surrogate models tested with one left out case each) covers all possible surrogate model of $n - 1$ cases plus the one test case combinations. Doing another iteration of this will result in exactly the same predicitons unless your classifier is non-deterministic.

In the case of a completely deterministic classifier the random number generator is not needed at all. If your classifier uses the random number generator, you can set the seed at the beginning of the cross validation.
If I'd want to be able to exactly reproduce arbitrary surrogate models, I'd log the state of the random number generator just before the call to the classifier rather than setting it.

Note that while the result of leave-one-out cross validation of a deterministic classifier will be exactly the same for each iteration, it is still subject to variance due to the finite number of tested cases and instability of the surrogate models.