I'm running some SVMs for a seminar and a friend of mine noted I should set a seed so my results don't change everytime I run the code. I was wondering why is that the case. If a different seed can induce different results, why should I trust SVMs at all?

Should I set a specific seed or is it ok to just set the first number that comes to my mind?

  • 5
    "No man ever steps in the same river twice, for it's not the same river and he's not the same man." -- Heraclitus. Any time you do something, the results differ a little. Why should you demand, then, that a statistical procedure be any different? What matters isn't that the result changes, but by how much and whether it makes any difference. See stats.stackexchange.com/search?q=seed+set+random for discussions of this issue. – whuber Nov 18 at 19:45
up vote 12 down vote accepted

tl;dr practically speaking, you can probably set the seed to anything you want (e.g. your birthday or phone number [although there are obvious privacy issues there :-)] or your lucky number); with some interesting caveats, you can use the same random number seed for most of your analyses (I often use 1001). In order to be useful, stochastic algorithms are generally insensitive to the random number seed.

the long answer

Classical statistical methods (t-test, ANOVA, regression etc.) are deterministic algorithms, but many modern algorithmic approaches include a stochastic component. (In between are methods like k-means clustering or expectation-maximization, which are intrinsically deterministic but are usually run from multiple randomly chosen starting points to mitigate their sensitivity to starting conditions.)

SVM need not be stochastic (e.g. the implementation in the e1071 package for R appears to be deterministic), but it is often implemented using stochastic gradient descent (SGD: e.g. see here) for computational reasons.

Methods that are using large ensembles of random samples from the data (e.g. bootstrapping, bagging, as well as SGD, which picks a different sample of the data at each update step) are effectively averaging across many samples, and are likely to be relatively insensitive to the random-number seed. Methods that are likely to be unstable with respect to the random-number seed (e.g. EM, k-means clustering) will generally have mechanisms built into the software that will automatically run several realizations and do something sensible with the results (i.e. average them), to make the method less sensitive.

This sensitivity is part of the information that you should know about a method before using it (along with some idea of its strengths and weaknesses, what meta-parameters it has that need to be tuned, etc.).

The best thing to do in the course of learning is to try some experiments - for a particular data set and model, try the same method with a handful of different random-number seeds and see how much the results vary!

  • +1. For a discussion of some of the issues associated with using arbitrary seeds, please see stats.stackexchange.com/questions/80407. – whuber Nov 18 at 21:35
  • As well, even for a deterministic approach it may be dependent on a starting point or the ordering of the data. – MotiN Nov 25 at 9:40
  • I did make the point about starting point above ... and I wasn't dealing with issues related to numerical instability in my answer, seems slightly tangential ... – Ben Bolker Nov 25 at 13:46

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.