I am performing experiments on randomized algorithms, and want to do it scientifically sound.

What relative standard error of mean (RSEM) is acceptable? I know that often RSEM<30% or RSEM<25% is used, but does that also apply for randomized algorithms?

What paper/book can I cite that give a reliability standard? This would optimally be a standard for the field of computer science or randomized algorithms, but a general reliability standard would be fine, too.

Details: I only found

  • $\begingroup$ This is a question for a different forum, I think, because "reliability" is a matter of the application and its sensitivity to errors, timing, or whatever else you use to measure reliability. It is not a matter of statistics or data analysis. Accordingly it would be surprising to find any general standard adopted within a broad field like computer science: that would be at best irresponsible and at worst dangerous. (Contemplate the consequences of using a generic standard for benchmarking real-time medical applications, for instance.) $\endgroup$ – whuber Nov 1 '13 at 16:40
  • $\begingroup$ Nice example, whuber. I agree that one fixed threshold makes little sense for a large field. On the other hand, the health sector is a general field too, and there is the overview document I cited that lists several reliability standards depending on specific domains. So maybe there is something similar for computer science or randomized algorithms? $\endgroup$ – DaveFar Nov 1 '13 at 17:51
  • $\begingroup$ Should this post be migrated to cs.stackexchange.com ? $\endgroup$ – DaveFar Nov 1 '13 at 17:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.